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Abstract 

In this paper, we have estabhshed a unified framework of multistage parametric estimation. 
We demonstrate that a wide spectrum of classical sequential problems such as point estimation 
with error control, bounded-width confidence intervals, interval estimation following hypoth- 
esis testing, construction of confidence sequences, can be cast in the general framework of 
random intervals. We have developed exact methods for the construction of such random 
intervals in the context of multistage sampling. Our sampling schemes are unprecedentedly 
efficient in terms of sampling effort as compared to existing sampling procedures. 
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1 Introduction 

Parameter estimation is a fundamental area of statistical inference, which enjoys numerous appli- 
cations in various fields of sciences and engineering. Specially, it is of ubiquitous significance to 
estimate, via sampling, the parameters of binomial, Poisson, hypergeometrical, and normal dis- 
tributions. In general, a parameter estimation problem can be formulated as follows. Let X be a 
random variable defined in a probability space (O, ^ ^ Pr). Suppose the distribution of X is deter- 
mined by an unknown parameter in a parameter space 0. In many applications, it is desirable 
to construct a random interval which includes Q with a prescribed level of confidence from random 
samples Xi,X2, • • • of X. This problem is so fundamental that it has been persistent issues of 
research in probability, statistics and other relevant fields (see, e.g., [181 [22l [23l [32l [35] [U] and the 
references therein). Despite the richness of literature devoted to such issues, existing approaches 
may suffer from the drawbacks of lacking either efficiency or rigorousness. Such drawbacks are 
due to two frequently- used methods of designing sampling schemes. The first method is to seek a 
worst-case solution based on the assumption that the true parameter Q is included in an interval 
[a, 6] C O. Since it is difficult to have tight bounds for the unknown parameter ^, such a worst-case 
method can lead to overly wasteful sample size if the interval [a, 6] is too wide. Moreover, if the 
true value of Q is not included in [a, 6], the resultant sample size can be misleading. The second 
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method is to employ asymptotic theories such as large deviations theory, Brownian motion theory, 
diffusion theory and nonlinear renewal theory in the design and analysis of sampling schemes (see, 
[T71 [281 EZl HOI 112] and the references therein) . Undoubtedly, asymptotic techniques may offer 
approximate solutions and important insight for the relevant problems. Since any asymptotic 
theory holds only if the sample size tends to infinity and, unfortunately, any practical sampling 
scheme must be of a finite sample size, it is inevitable for an asymptotic method to introduce 
unknown error in the resultant approximate solution. 

In view of the limitations of existing approaches of parametric estimation, we would like to 
propose a new framework of multistage estimation. The main characteristics of our new estimation 
methods is as follows: i) No information of the parameter 6 is required; ii) The sampling schemes 
are globally efficient in the sense that the average sample number is almost the same as the exact 
sample size computed as the true value of 9 were available; iii) The prescribed level of confidence 
is rigorously guaranteed. Our new estimation techniques are developed under the spirit that 
parameter estimation, as an important branch of statistical inference, should be accomplished 
with minimum cost in sampling and absolute rigorousness in quantifying uncertainty. 

The remainder of the paper is organized as follows. In Section 2, we present our general theory 
for the design and analysis of multistage sampling schemes. Especially, we have established 
a general theory on coverage probability of random intervals which eliminates the necessity of 
exhaustive computation of coverage probability for designing sampling schemes. In Section 3, 
we introduce powerful techniques such as bisection coverage tuning, consecutive-decision-variable 
bounding, recursive computation, adaptive maximum checking, domain truncation and triangular 
partition that are crucial for a successful design of a multistage sampling scheme. In Section 4, 
we present sampling schemes for estimation of binomial parameters. In Section 5, we discuss the 
multistage estimation of Poisson parameters. In Section 6, we consider the estimation of means of 
bounded variables. In Section 7, we address the problem of estimating the proportion of a finite 
population. We consider the estimation of normal mean with unknown variance in Section 8. In 
Section 9, we discuss the estimation of the parameter of an exponential distribution. In Section 
10, we propose our exact methods for the construction of bounded-width confidence intervals. 
In Section 11, we discuss the interval estimation following hypothesis testing. In Section 12, we 
consider the exact construction of confidence sequences. In Section 13, we address the problem of 
multistage linear regression. In Section 14, we investigate the multistage estimation of quantile. 
Section 15 is the conclusion. The proofs of all theorems are given in Appendices. 

Throughout this paper, we shall use the following notations. The set of integers is denoted 
by Z. The set of positive integers is denoted by N. The element of matrix A in the z-th row and 
j-th column is denoted by The ceiling function and fioor function are denoted respectively 

by [.] and [.J (i.e., \x\ represents the smallest integer no less than x; \x\ represents the largest 
integer no greater than x). The notation sgn(x) denotes the sign function which assumes value 1 
for X > 0, value for x = 0, and value —1 for x < 0. The gamma function is denoted by r(.). For 
any integer m, the combinatoric function with respect to integer z takes value r(z+i)r(m-z+i) 
for z < m and value otherwise. The left limit as e tends to is denoted as lim^^o- The notation 
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"<J=^>" means "if and only if". The expectation of a random variable is denoted by E[.]. The 
notation Pr{. | 9} denotes the probability of an event associated with random samples Xi,X2, ■ ■ ■ 
parameterized hy 9 £ Q, where 9 may be dropped if it can be done without introducing confusion. 
The parameter 9 in Pr{. | 9} may be dropped whenever this can be done without introducing 
confusion. The cumulative distribution function of a Gaussian random variable is denoted by <!?(.). 
For a S (0,1), let Za and denote, respectively, the 100(1 — a)% percentiles of a standard 
normal distribution and a Student t-distribution of n degrees of freedom. For a E (0, 1), let 
Xn,a denote the 100a% percentile of a chi-square distribution of n degrees of freedom. In the 
presentation of our sampling schemes, we need to use the following functions: 
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for < z < 1 and 6 ^ (0, 1) 

In i^f for z G (0, 1) and 9 £ (0, 1) 
forz = 0and6'e (0,1), 
for z = 1 and 9 G (0, 1), 
for z e [0, 1] and 9 ^ (0, 1) 

i^f for z e (0,1) and 6* e (0,1), 
for z = 1 and 9 e (0, 1), 
for z = and 9 e (0, 1), 
for z e [0, 1] and 9 ^ (0, 1) 

for z > and 6* > 0, 
for z = and > 0, 
for z > and 6* < 0. 



In the design of multistage sampling schemes, we shall use a descending sequence C^, £ € Z such 
that Co = 1 and 1 < inf^g^ — ^^P^ez < oo to define sample sizes. Throughout the 
remainder of this paper, 6 and C are reserved, respectively, for the "confidence parameter" and 
the "coverage tuning parameter", where these concepts will be illustrated later. It is assumed 
that < 5 < 1 and < C < ^. The other notations will be made clear as we proceed. 
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2 General Theory 



In this section, we shall discuss the general theory of multistage estimation. A central theme of 
our theory is on the reduction of the computational complexity associated with the design and 
analysis of multistage sampling schemes. 

2.1 Basic Structure of Multistage Estimation 

In our proposed framework of multistage estimation, a sampling process consists of s stages, where 
s can be a finite number or infinity. The continuation or termination of sampling is determined 
by decision variables. For the £-th stage, a decision variable = ^^{Xi, • • • , X^) is defined in 
terms of samples Xi, • • • ,Xn^, where is the number of samples available at the £-th stage. It 
should be noted that can be a random number, depending on specific sampling schemes. The 
decision variable Dg assumes only two possible values 0, 1 with the notion that the sampling 
process is continued until Di = 1 for some £ G {1, • • • , s}. Since the sampling must be terminated 
at or before the s-th stage, it is required that Dg = 1. For simplicity of notations, we also define 
Dg = for £ < 1 and Dg = 1 for i > s throughout the remainder of the paper. Let I denote the 
index of stage when the sampling is terminated. Then, the sample number when the sampling is 
terminated, denoted by n, is equal to nj. Since a sampling scheme with the above structure is 
like a multistage version of the conventional fixed-size sampling procedure, we call it multistage 
sampling in this paper. 

As mentioned earlier, the number of available samples, n^, for the £-th stage can be a random 
number. An important case can be made in the estimation of the parameter of a Bernoulli random 
variable X with distribution Pr{X = 1} = 1 — Pr{X = 0} = p G (0, 1). To estimate p, we can 
choose a sequence of positive integers 71 < 72 < • • • < 7s and define decision variables such that 
-Df is expressed in terms of i.i.d. samples Xi, • • • , X^^ of Bernoulli random variable X, where 
is the minimum integer such that ^^^=1 -^i ~ It i = 1, - ■ ■ ,s. A sampling scheme with such a 
structure is called a multistage inverse binomial sampling, which is a special class of multistage 
sampling schemes and is a multistage version of the inverse binomial sampling (see, e.g., \25\ I26j 
and the references therein). 

If the sample sizes of a multistage sampling scheme is desired to be deterministic, the following 
criteria can be applied to determine the minimum and maximum sample sizes: 

(I) The minimum sample size ni guarantees that {Di = 1} is not an impossible event. 

(II) The maximum sample size guarantees that {Ds = 1} is a sure event. 

For the purpose of reducing sample number, the minimum and maximum sample sizes should 
be as small as possible, while satisfying criteria (I) and (II). 
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2.2 Truncated Inverse Sampling 

It should be noted that the conventional single stage sampling procedures can be accommodated in 
the general framework of multistage sampling. A common stopping rule for single stage sampling 
procedures is that "the sampling is continued until the sample sum reach a prescribed threshold 
7 or the number of samples reach a pre-specified integer m" . Such a sampling scheme is referred 
to as truncated inverse sampling, for which we have derived the following results. 

Theorem 1 Let 7 > 1, < Sa < £r < ^ and p* = f^. Let Xi, X2, ■ ■ ■ be a sequence of i.i.d. 
random variables such that < < 1 and K[Xi] = /-f G (0, 1) for any positive integer i. Let n he 
a random variable such that ^^Yl,^=i <1 ^ 2"=! "^«} ^ '^^'"^ event. Let m = min{n,TO}, where 
m is a positive integer. The following statements hold true. 

(I) Pr{| J -^L\<eti}>l-5 and Pr{|^ - ^Ji\ < eii} > I - 5 provided that 7 > (/++)t'ggl, - 

(II) Pr {1^ - ^-^\ < Eq or 1^ - ^1 < e^//} > 1 - 5 provided that p* + £a < \, 1 > 7 > 
^ > '"(-^/z) and m > '"(^/'^ 

(III) If Xi,X2, • • • are i.i.d. Bernoulli variables, then Pr{|^ — ^| < or |^ — mI < ^j-m} > 1^*^ 
provided that + < i 7 > ^,lp*+!lp*) and m > 

The proof of Theorem [T] can be found in O II]- 



2.3 Random Intervals 

A primary goal of multistage sampling is to construct, based on samples of X, a random interval 
with lower limit ^(Xi, • • • ,Xa) and upper limit '^{Xi, ■ ■ ■ ,X„) such that, for a priori specified 
confidence parameter 5, 

Pr{^(Xi, ■■■ ,Xn)<e< ^{Xu • • • I 0} > 1 - 5 

for any G 0. For the i-ih. stage, an estimator 0^ for 9 can be defined in terms of samples 
Xi , • • • , X-^i . Consequently, the overall estimator for 9, denoted by 0, is equal to 9i . In many cases, 
^(Xi, • • • , X-ni,) and ^ [Xi, • • • , X^^,) can be expressed as a function of 61 and n^. For simplicity 
of notations, we abbreviate ^(Xi,--- ,XnJ and ^{Xi,--- ,Xn^) as ^(0^,nf) and '^{Oi^ni) 
respectively. Accordingly, _Sf(Xi,--- ,Xn) and ^(Xi,--- ,Xn) are abbreviated as ^(0,n) and 
'^(0,n). In the special case that the lower and upper limits are independent of n, we will drop 
the argument n for further simplification of notations. 

In the sequel, we shall focus on the construction of random intervals of lower limit J^{0, n) 
and upper limit '^(0,n) such that Pr{^(0,n) < < '^(0, n) | 6*} > 1 - 5 for any G G. Such 
a framework is general enough to address a wide spectrum of traditional problems in parametric 
estimation. First, it is obvious that the problem of interval estimation following a hypothesis 
test can be cast in this framework. Second, the issue of error control in the point estimation of 
parameter 9 can be addressed in the framework of random intervals. Based on different error 
criteria, the point estimation problems are typically posed in the following ways: 
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(i) Given a priori margin of absolute error e > 0, construct an estimator for 9 such that 
Pr{|0 - 6l| < e I 61} > 1 - (5 for any G G. 

(ii) Given a priori margin of relative error e € (0, 1), construct an estimator 6 for 6 such that 
Pr{\d -e\< e\e\ I 0} > 1 - (5 for any 6' G 6. 

(iii) Given a priori margin of absolute error Sa > and margin of relative error Sr G [0, 1), 
construct an estimator for 9 such that Pr{|0 — 9\ < Sa or \6 — 9\ < er\9\ \ 9} > 1 — 5 for any 
9eQ. 

Clearly, problem (iii) can be reduced to problems (i) and (ii) by, respectively, setting Sr = 
and Ea = 0. As can be seen from Appendix IA.1|, putting 

^{d,n) = mmle - ea, ^ >, ^^(0, n) = max i + e^, ^ 



1 + Sgn(0) Er } [ 1 - Sgn(0) Er J 

we can show that 

{\d-9\ <EaOr\d-9\ <Er\9\} = {^{d,n) <9<^(6,n)}. (1) 

This implies that problems (i)-(iii) can be accommodated in the general framework of random 
intervals. 

Third, the framework of random intervals accommodates an important class of problems con- 
cerned with the construction of bounded-width confidence intervals. The objective is to construct 
lower confidence limit J^{9, n) and upper confidence limit (6, n) such that {6, n)—^{d, n)| < 
2e for some prescribed number e > and that Pr{.j5f (0, n) < 9 < ^ {6,n) \ 9} > 1 — 5 for any 
9 £ Q. Obviously, this class of problems can be cast into the framework of random intervals. 

In order to construct a random interval of desired level of confidence, our global strategy is to 
construct a sampling scheme such that the coverage probability Pr{^(0,n) < 9 < {6,n) \ 9} 
can be adjusted by some parameter This parameter <^ is referred to as "coverage tuning 
parameter". Obviously, the coverage probability is a function of the unknown parameter 9. In 
practice, it is impossible or extremely difficult to evaluate the coverage probability for every value 
of 9 in the parameter space. Such an issue presents in the estimation of binomial parameters, 
Poisson parameters and the proportion of a finite population. For the cases of estimating binomial 
and Poisson parameters, the parameter spaces are continuous and thus the number of parametric 
values is infinity. For the case of estimating the proportion of a finite population, the number of 
parametric values can be as large as the population size. To overcome the difficulty associated 
with the number of parametric values, we have developed a general theory of coverage probability 
of random intervals which eliminates the need of exhaustive evaluation of coverage probabilities 
to determine whether the minimum coverage probability achieves the desired level of confidence. 
In this direction, the concept of Unimodal- Likelihood Estimator, to be discussed in the following 
subsection, play a crucial role. 
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2.4 Unimodal-Likelihood Estimator 

The concept of maximum-likelihood estimator (MLE) is classical and widely used in numerous 
areas. However, a MLE may not be unbiased and its associated likelihood function need not be 
monotone. For the purpose of developing a rigorous theory on coverage probability of random 
intervals, we shall introduce the concept of unimodal-likelihood estimator (ULE) in this paper. For 
samples Xi, • • • , of random length m with Xi parameterized by 6, we say that the estimator 
ip{Xi,--- ,Xni) is a ULE of if is a multivariate function such that, for any observation 
(xi,--- ,Xm) of (Xi,--- the likelihood function is non-decreasing with respect to 9 no 

greater than <^(xi, • • • ,Xm) and is non-increasing with respect to 9 no less than '^{xi, ■ ■ ■ ,Xm)- 
For discrete random samples Xi, ■ ■ ■ ,Xm, the associated likelihood function is Pr{Xi = Xi, i = 
1, • • • , m I 9}. For continuous random samples Xi, • • • , X^, the corresponding likelihood function 
is, fxi,-,Xjn{^ij ■ ■ ■ ,Xm,9), the joint probability density function of random samples Xi, ■ ■ ■ ,Xm- 
We emphasize that a MLE may not be a ULE and that a ULE may not be a MLE. In contrast 
to a MLE, a ULE can assume values not contained in the parameter space. 

Clearly, for the cases that X is a Bernoulli or Poisson variable, ip{Xi,--- ,XnJ = n] — ' 
a ULE of 9 at the i-th stage. As another illustration of ULE, consider the multistage inverse 
binomial sampling scheme described in Section [2.11 For £ = 1, • • • s, a ULE of p can be defined as 
Pe = At the termination of sampling, the estimator, p = Pi, of p is also a ULE. 

2.5 Principle of Construction of Sampling Schemes 

In this subsection, we shall discuss the fundamental principle for the design of multistage sampling 
schemes. We shall address two critical problems: 

(I) Determine sufficient conditions for a multistage sampling scheme such that the coverage 
probability Pr{Jf (0, n) < 9 < 'W {6,n) \ 9} can be adjusted by a positive number 

(II) For a given sampling scheme, determine whether the coverage probability Pr{^(0,n) < 
9 < '^{0,n) I 9} is no less than 1 - 5 for any 9 e @. 

To describe our sampling schemes, define cumulative distribution functions (CDFs) 

{PT{de<z\e} ioi-eee, iPT{de>z\e} foreee, 

1 ioie<0, Gg^(z,0)=io for 6 < 6, 

for 61 > ^ [ 1 for 6* > e 

for i = 1, • • • , s, where 9 and 9 represent the infinimum and supremum of G respectively, 
and z assumes values in the support of O^. For Theorem [2] and Corollary [1] to be presented 
in the sequel, we make a common assumption that the relevant random intervals satisfy {9 < 
■^(Oi,ne) <9}^ {^(ee,iie) G 6} and {^< ^(0^,n£) < ^} C {'^(0^,n^) e 6} for £ = 1, • • • , s. 

Let 6i £ (0, 1), i = 1, • • • ,s. For sampling schemes of structure described in Section 12. H we 
have the following results on the coverage probability of random intervals. 
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Theorem 2 Suppose that a multistage sampling scheme satisfies the following requirements: 
(i) Fori=l,---,s,deisa ULE of 9. 

(a) For i = 1, - ■ ■ ,s, {^{6i, n^) <0i<^ {6^, n^)} is a sure event. 

(Hi) {De = 1} C {Fg^ (g,,^(0,,n,)) < (S^, Gg^ ^(0^ , n,)) < (Sf} for £ = 1, ■ ■ ■ , s. 

(iv) {Ds = 1} is a sure event. 

Then, 

s s 

Pv{^(e,n) >9\e} < J^Pr{if(0,,n,) > 9, De = 1 \ 9} < (^,6^, 

e=i 1=1 

s s 

Pr{^(?,n) <9\e]< J];Pr{^(0^,nf) < 9 , = I \ 9] < Q^.^^^ 

i=i 1=1 

s 

Pr{^(?,n) <9 < ^(0,n) \9}>l-2C^5i 

1=1 

for any 9 G Q. 

See Appendix [B] for a proof. Theorem [2] addresses the first problem posed at the beginning 
of this subsection. It tells how to define a stopping rule such that the coverage probability of 
the random interval can be bounded by a function of C and • If Yle=i bounded with 

respect to C,, then, the coverage probability can be "tuned" to be no less than the prescribed level 
1 — 5. This process is referred to as "coverage tuning" , which will be illustrated in details in the 
sequel. The intuition behind the definition of the stopping rule in Theorem [2] is as follows. 

At the £-th stage, in order to determine whether the sampling should stop, two tests are 
performed based on the observations of 0^, ^(0^,n£) and ^ [d^jUf), which are denoted by 

Li and Ui respectively. The first test is : 9 < Ue versus J^i : 9 > Ug, and the second 
test is : 9 < Li versus : 9 > Lg. Hypothesis J% is accepted if F-^^{i}i,Ui) < C^i, and 
is rejected otherwise. On the other side, hypothesis J^q is rejected if Gg^{'&i,L£) < (dg, and is 
accepted otherwise. If Jifo is accepted and is rejected, then, the decision variable Di assumes 
value 1 and accordingly the sampling is terminated. Otherwise, Di assumes value and the 
sampling is continued. It can be seen that, if (^5i is small, then J% and are accepted with 
high credibility and consequently, < 9 < is highly likely to be true. Therefore, by making 
C ^£=1 sufficiently small, it is possible to ensure that the coverage probability of the random 
interval is above the desired level. 

For simplicity of stopping boundary, we have established multistage sampling schemes by 
virtue of Theorem [2] as follows. 

Corollary 1 Suppose that a multistage sampling scheme satisfies the following requirements: 
(i) Fori=l,---,s,deisa ULE of 9. 
(a) For i = 1, • • • ,s, E[e*^^] exists for any real number t. 
(Hi) For i = 1, - ■ ■ ,s, {.^{Oe, n^) < 0^ < (Oi, n^)} is a sure event. 
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(iv) {De = 1} C 1^4+ (0,,if(gf,n,)) < (Si, '^f n^)) < (S^'j for i = 1,- ■ ■ ,s, where 
.) and "lo^ .) are functions such that 

'inft>o e-*^ E[e*^^] for OeQ, f inft<o e"*^ E[e*^^] /or 6* e 6, 

'^+(z,6') = <( fore < 9, 'tff{z,0) = ll for 9 < 9, 

for9>9 [o for9>9 

(v) {Ds = 1} is a sure event. 
Then, 



Pr{^(g,n) >e\e}< ^Pr{if(0,,n,) >e,D, = i\e} < C^5£, 

£=1 £=1 

s s 

Pr{^(0,n) < I 0} < ^Pr{^(0£,n^) < 0, = 1 | 0} < C 

£=1 £=1 

s 

Pr{^(^, n) < < ^(0, n) I > 1 - 2C ^ 5f 



for any 9 £ Q. 



To establish Corollary [H it suffices to show that the assumption (iv) of Corollary [T] implies 
the assumption (iii) of Theorem [2l which can be seen from Chernoff bounds 

F- (z,e) = Pi{di <z\e}< inf e*^ E[e~*^*] = inf e"*^ E[e*^^l = ^7(^,0), 
Gg (z, 9) = Fr{de > z \ 9} < inf e"*^ E[e*^^] = 0) 

for G and z assuming values from the support of 6i. 

Now, we turn to consider the second problem posed at the beginning of this subsection. For 
the sampling schemes of structure described in Section [2.11 we have the following results regarding 
the coverage probability of random intervals. 

Theorem 3 Let Xi, X2, ■ ■ ■ be a sequence of identical samples of discrete random variable X 
parameterized by 9 £ Q. For i = 1, ■ ■ ■ , s, let 6£ = Lp{Xi,--- ,X,^^) be a ULE of 9. Define 
estimator = 9i, where I is the index of stage when the sampling is terminated. Let ^(., .) and 
^(.,.) be bivariate functions such that {^{6,n) <6< '^{6,n)} is a sure event. Let [a,b] be a 
subset of Q. Let L_^ denote the intersection of [a,b] and the support o/^(0,n). Let La// denote 
the intersection of [a, b] and the support of {0, n). Let S" be an event determined by the random 
tuple (Xi, • • • ,Xn). The following statements hold true: 

(I) Both Pr{^(0,n) > 9 and S occurs \ ^} and Pr{^(0, n) > 9 and S occurs \ 9} are 
no- decreasing with respect to 9 in any open interval with endpoints being consecutive distinct 
elements of L^ U {a, 6}. Moreover, both the maximum o/Pr{Jf(0,n) > 9 and S occurs \ 0} and 
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the supremum o/Pr{^(0,n) > 6 and £ occurs \ 0} with respect to 6 £ [a,b] are equal to the 
maximum o/Pr{_Sf (0, n) > and <§ occurs \ 6} for 6 G U {a, 6}. 

(II) Both Pr{'^(0,n) < 9 and £ occurs \ 6] and Pr{^(0,n) < 9 and occurs \ 0] are 
non-increasing with respect to 6 in any open interval with endpoints being consecutive distinct 
elements of If/ U{a,b}. Moreover, both the maximum o/Pr{'^(0,n) < 9 and S" occurs \ 9} and 
the supremum of Prj"^ (6, n) < 9 and S occurs \ 0} with respect to 9 £ [a, b] are equal to the 
maximum of Pr{'^ (6, n) < 9 and £ occurs \ 9} for 9 £ L;// L) {a, b}. 

(III) If {^{d,n) > a} C {0 > b}, then Pr{J^(0,n) > b and S occurs \ a} < Pr{^(0, n) > 
9 and S" occurs \ 6} < Pr{^(0, n) > a and S occurs \ 6} and Pr{^(0, n) > h and § occurs \ a} < 
Pr{^(0,n) > 6 and S occurs \ 6} < Pr{^(0,n) > a and S" occurs \ b} for any 9 € [ci,b]. Similarly, if 
{^(0, n) < 6} C {0 < a}, then Pr{'^(0,n) < a and S' occurs \ 6} < Pr{^(0,n) < 9 and S occurs \ 
6} < Pr{'^(§,n) < b and S occurs \ a} and Pr{'^(0,n) < a and S occurs \ 6} < Pr{^(0,n) < 
6 and S occurs \ 9} < Pr{^(0, n) < b and § occurs \ a} for any 9 G [a, 6] . 

See Appendix O for a proof. In Theorem El we have used the concept of support in probability 
theory. The support of a random variable Z refers to {Z{u) : u S $7}, which is the set of all 
possible values of Z. 

Based on Theorem [3] in the special case that is a sure event, two different approaches can 
be developed to address the second problem proposed at the beginning of this subsection. 

First, as a consequence of statements (I) and (II) of Theorem[3l it is true that Pr{^(0,n) < 
9 < {d ,n) \ 9} > 1 - S for any 9 G [a, b] provided that 

Pi{9<^(e,ii)\9}<-, y9€l^U{a,b}, 

PT{9>^{d,ii)\9}<-, VeG%U{a,6}. 

As can be seen from the proofs of Theorems 1 and 2, under certain conditions, the probabilities 
Fr{9 < J^'{6, n) | 9} and Pt{9 > ^ {6, n) | 9} can be adjusted by C- Hence, it is possible to obtain 
appropriate value of C,, without exhaustive evaluation of probabilities, such that Pr{_5f(0,n) < 
^ < '^(0, n) I ^} > 1 - 5 for any 9 G [a, b]. 

Second, statements (III) and (IV) of Theorem [3] will be used to develop Adaptive Maximum 
Checking Algorithm in Section [3^31 to determine an appropriate value of coverage tuning parameter 

C- 

In the special case that the number of stages s is equal to 1 and that the sample number is a 
deterministic integer n, we have the following results. 

Theorem 4 Let Xi,X2, - ' ' be a sequence of identical samples of discrete random variable 
X which is parameterized by 9 £ 0. Let 9 = (p{Xi,--- ,Xn) be an estimator of 9 such that 
Pr{9 < < 9 \ 0} is a continuous and unimodal function of 9 £ Q for any numbers 9 and 9. Let 
^{.) and ^{■) be functions such that there exist intervals Ig and I'g of real numbers satisfying 
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{^(6) <e< ^(0)} = {e e le} and <e < '^(6)} = {0 e I'g} for any 9 G [a,b], where 

[a,b] is a subset of Q. Let ly denote the intersection of [a,b] and the support of ^(6). Let La// 
denote the intersection of [a, 6] and the support of'^{0). The following statements hold true: 

(I) The minimum of Pr{^{6) < 9 < '^[9) \ 0} with respect to 9 G [a, 5] is attained at the 
discrete set {a,b} U ^ ^l; (W The infimum o/ Pr{^(0) < 9 < '^{6) \ 9} with respect to 
9 G [a,b] equals the minimum of the set {C{a), C{b)} U {Cu{9) : 9 G ^u} U {Cl{9) : 9 G ^l}, 
where C{9) = Pr{^(0) < 9 < ^(0) | 9}, Cu{9) = Pr{if(0) < 9 < '^{6) \ 9} and Cl{9) = 
Vi{J^{9) < 9 < '^{6) I 9}; (III) For both open and closed random intervals with lower limit 
.Sf{0) and upper limit '^(6), the coverage probability is continuous and unimodal for 9 G {9', 9"), 
where 9' and 9" are any two consecutive distinct elements of {a,b} U U ^l- 

The proof of Theorem H] can be found in [5] . 



2.6 Multistage Sampling without Replacement 

It should be noted that the theories in preceding discussion can be applied to the multistage 
estimation of the proportion of a finite population, where the random samples are dependent if a 
sampling without replacement is used. Consider a population of N units, among which there are 
pN units having a certain attribute, where p G © = {-^ : M = 0, 1, • • • , N}. In many situations, 
it is desirable to estimate the population proportion p by sampling without replacement. The 
procedure of sampling without replacement can be precisely described as follows: 

Each time a single unit is drawn without replacement from the remaining population so that 
every unit of the remaining population has equal chance of being selected. 

Such a sampling process can be exactly characterized by random variables Xi, - ■ ■ , Xn defined 
in a probability space (0, Pr) such that Xi assumes value 1 if the i-th sample has the attribute 
and assumes value otherwise. By the nature of the sampling procedure, it can be shown that 

pr{x. (^!:~^_) '^^^^^ I [ (^.r^ (^) 

for any n G {!,••• , -/V} and any Xj G {0,1}, i = 1, • • • ,n. Clearly, for any n G {1, • • • ,-/V}, 
the sample mean — - is unbiased but is not a MLE for p £ Q. However, we have shown in 
Appendix |D] the following result: 

Theorem 5 For any n G {1, • • • , N}, ^'=^ ^' is a ULE for p G 0. 

Based on random variables Xi, • • • ,X7v, we can define a multistage sampling scheme in the 
same way as that of the multistage sampling described in Section 12.11 More specially, we can 
define decision variables such that, for the £-th stage, is a function of Xi, • • • , Xn^, where the 
random variable is the number of samples available at the ^-th stage. For £ = 1, • • • ,s, an 
estimator of p at the ^-stage can be defined as = ^'=^ — -. Letting I be the index of stage when 
the sampling is terminated, we can define an estimator for p as p = = — ^, where n = nj 
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is the sample size at the termination of samphng. A samphng scheme described in this setting is 
referred to as a multistage sampling without replacement in this paper. Regarding the coverage 
probabihty of random intervals, we have the following results which are direct consequence of 
Theorems [3] and \5\ 

Corollary 2 Let ^(.,.) and ^{.,.) be bivariate functions such that {Jf (p, n) <p< '^(p, n)} 
is a sure event and that both N^(p, n) and (p, n) arc integer-valued random variables. Let 
[a,b] be a subset of Q . Let Ljc denote the intersection of {a, b) and the support of ^{p,n). Let If/ 
denote the intersection of {a,b) and the support o/^(p, n). The following statements hold true: 

(I) Pr{^(p, n) > p \ p} is non- decreasing with respect to p & Q in any interval with endpoints 
being consecutive distinct elements of I_^U{a,b} . Moreover, the maximum o/Pr{^(p, n) > p \ p} 
with respect to p £ [a, b] is achieved at Ij^ U {a, b}. 

(II) Vr{'W (p, n) < p \ p} is non-increasing with respect to p £ Q in any interval with endpoints 
being consecutive distinct elements of Lp/L){a,b} . Moreover, the maximum o/Pr{'^(p, n) < p \ p} 
with respect to p £ [a, b] is achieved at Lp/ U {a, b}. 

(III) If {^{p,n) > a} C {p > b}, then Pr{^(p,n) > 6 | a} < Pr{^(p,n) > p \ p} < 
Pr{^(p, n) > a I 6} for any p G [a, b] . Similarly, if \^ (p, n) < 6} C {p < a}, then Pr{'^ (p, n) < 
a I 5} < Pr{'^ (p, vl) <p\p\ < Pr{^ (p, n) < 6 | a} for any p G [a, 6] . 

In the special case that the number of stages s is equal to 1 and that the sample number is a 
deterministic integer n, we have the following results. 

Theorem 6 Let [a, 6] be a subset ofQ. Suppose that.^[.) and^{.) are non- decreasing functions 
such that both N^{p) and N'^ (p) are integer-valued random variables. Then, the minimum of 
Pr{^(p) < p < ^ (p) I p} with respect to p G [a, b] is attained at a discrete set Ijjl which is the 
union o/ {a, 6} and the supports of ^{p) and ^(p). Moreover, Pr{^(p) < p < ^(p) | p} is 
unimodal with respect to p in between consecutive distinct elements of Iul- 

The proof of Theorem [6] can be found in [5]. 
2.7 Asymptotically Unbiased Estimators of Mean Values 

Some important distributions are determined by the mean values of associated random variables. 
Familiar examples are binomial distribution, Poisson distribution, normal distribution, and ex- 
ponential distribution. To estimate the expectation, ^, of a random variable X based on i.i.d. 
samples Xi , X2 , • • • , we can use a multistage sampling scheme with a structure described in Sec- 
tion 12.11 Specially, an estimator of /i can be defined as the sample mean p, = — -, where 
n is the sample number at the termination of sampling. To justify that the estimator /x is su- 
perior than other estimators, we shall show its asymptotic unbiasedness and relevant properties. 
For a multistage sampling scheme with deterministic sample sizes 7ii < 712 < ■ ■ ■ < n^, we have 
established the following general results. 
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Theorem 7 Suppose that inff>o ^^^^ is greater than 1. The following statements hold true. 

(I) If X has a finite variance, t/ien E[/x — E|/x — and E|/x — ^p tend to as the minimum 
sample size tends to infinity. 

(II) If X is a bounded random variable, then E[/x — fi] and E|/i — ^j'^, k = 1,2, •• • tend to 
as the minimum sample size tends to infinity. 

See Appendix lEl for a proof. 

3 Computational Machinery 

3.1 Bisection Coverage Tuning 

To avoid prohibitive burden of computational complexity in the design process, we shall focus on 
a class of multistage sampling schemes for which the coverage probability can be adjusted by a 
single parameter ^. Such a parameter is referred to as the coverage tuning parameter in this 
paper to convey the idea that C is used to "tune" the coverage probability to meet the desired 
confidence level. As will be seen in the sequel, we are able to construct a class of multistage 
sampling schemes such that the coverage probability can be "tuned" to ensure prescribed level 
of confidence by making the coverage tuning parameter sufficiently small. One great advantage 
of our sampling schemes is that the tuning can be accomplished by a bisection search method. 
To apply a bisection method, it is required to determine whether the coverage probability for a 
given C is exceeding the prescribed level of confidence. Such a task is explored in the following 
subsections. 

3.2 Consecutive-Decision- Variable Bounding 

One major problem in the design and analysis of multistage sampling schemes is the high- 
dimensional summation or integration involved in the evaluation of probabilities. For instance, a 
basic problem is to evaluate the coverage probabilities involving 9 and n. Another example is to 
evaluate the distribution or the expectation of sample number n. Clearly, 6 depends on random 
samples Xi, • • • , X^- Since the sample number n can assume very large values, the computational 
complexity associated with the high-dimensionality can be a prohibitive burden to modern com- 
puters. In order to break the curse of dimensionality, we propose to obtain tight bounds for those 
types of probabilities. In this regard, we have 

Theorem 8 let /^(., .) be a bivariate function. let ^ be a subset of real numbers. Then, 

s 

Pr \w{d, n) e ^1 < ^ Pr \w{9(, ng) e ^, -Df = 1 and = for max(l, ^ - r) < j < , 

s 

Pr 1^(0, n) G > 1 - ^ Pr , n^) ^ ^, Di = l and = for max(l, £ - r) < j < ^| 

1=1 
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for < r < s. Moreover, 

Fr{l > £} < Fr{Di, = 0, Dj = for max(l,£ - r) < j < i}, 

t 

Pr{/ > ^} > 1 - ^ Vi{Dj = 1, A = for max(l, j - r) < i < j} 

i=i 

for 1 < I < s and < r < s. Furthermore, if the number of available samples at the l-th stage is 
a deterministic number ni for 1 < £ < s, then E[n] = ni + X]|=i I'^^+i ~ ''^^) Pr{' > £}■ 

See Appendix [F] for a proof. As can be seen from Theorem [HI the bounds are constructed by 
summing up probabihstic terms involving one or multiple consecutive decision variables (CDV). 
Such general technique is referred to as CDV bounding. A particular interesting special case of 
CDV method is to construct bounds with every probabilistic term involving consecutive decision 
variables (i.e., r = 1 in Theorem [8]). Such method is referred to as double-decision-variable or 
DDV bounding for brevity. Similarly, the bounds with each probabilistic term involving a single 
decision variable are referred to as single- decision-variable bounds or SDV bounds (i.e., r = 
in Theorem [8]). Our computational experiences indicate that the bounds in Theorem [8] become 
very tight as the spacing between sample sizes increases. As can be seen from Theorem [8l DDV 
bounds are tighter than SDV bounds. Needless to say, the tightness of bounds is achieved at the 
price of computational complexity. The reason that such bounding methods allow for powerful 
dimension reduction is that, for many important estimation problems, Di-i, De and Oi can be 
expressed in terms of two independent variables U and V. For instance, for the estimation of a 
binomial parameter, it is possible to design a multistage sampling scheme such that Di-i, Di and 
6i can be expressed in terms oiU = Yl!^=i -^i ^ = Y17=ni i+i -^i- double decision 

variable method, it is evident that U and V are two independent binomial random variables and 
accordingly the computation of probabilities such as Pt{W{9, n) G ^} and Pr{i > £} can be 
reduced to two-dimensional problems. Clearly, the dimension of these computational problems 
can be reduced to one if the single-decision-variable method is employed. As will be seen in 
the sequel, DDV bounds can be shown to be asymptotically tight for a large class of multistage 
sampling schemes. Moreover, our computational experiences indicate that SDV bounds are not 
very conservative. 

3.3 Adaptive Maximum Checking 

A wide class of computational problems depends on the following critical subroutine: 

Determine whether a function C{6) is smaller than a prescribed number 5 for every value of 
9 in interval [6, 6] . 

In many situations, it is impossible or very difficult to evaluate C{9) for every value of 9 in 
interval [9,9], since the interval may contain infinitely many or an extremely large number of 
values. To overcome such an issue of computational complexity, we shall propose an Adaptive 
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Maximum Checking Algorithm, abbreviated as AMCA, to determine whether the maximum of 
C{6) over [9,6] is less than 6. The only assumption required for our AMCA is that, for any 
interval [a,b] C \6_,9], it is possible to compute an upper bound C{a,b) such that C{9) < C{a,b) 
for any 9 £ [a, b] and that the upper bound converges to C{9) as the interval width b — a tends to 
0. 

Our backward AMCA proceeds as follows: 

• Choose initial step size d > r]. 

• Let F ^ 0, T ^ and 6 ^ ^. 

• While F = T = 0, do the following: 

- Let st and £ 2; 

— While st = 0, do the following: 

* Let ^ ^ £ - 1 and d ^ d2^. 

* If 6 - d > ^, let a ^ 6 - d and T ^ 0. Otherwise, let a and T 1. 

* If C{a, b) < 6, let st ^ 1 and b ■(^ a. 

* If d < r/, let st ^ 1 and F ^ 1. 

• Return F. 

The output of our backward AMCA is a binary variable F such that "F = 0" means ^^C{9) < 

5" and "F = 1" means ^^C{9) > An intermediate variable T is introduced in the description of 

AMCA such that "T = 1" means that the left endpoint of the interval is reached. The backward 

AMCA starts from the right endpoint of the interval (i.e., b = 9) and attempts to find an interval 

[a, b] such that C{a, b) < 6. If such an interval is available, then, attempt to go backward to find 

the next consecutive interval with twice width. If doubling the interval width fails to guarantee 

C{a, b) < 6, then try to repeatedly cut the interval width in half to ensure that C{a, b) < 5. If the 

interval width becomes smaller than a prescribed tolerance r], then AMCA declares that "F = 1" . 

For our relevant statistical problems, if C{9) > 6 for some 9 G [9,9], it is sure that "F = 1" will 

be declared. On the other hand, it is possible that "F = 1" is declared even though C{9) < 5 for 

any 9 € [9,9]. However, such situation can be made extremely rare and immaterial if we choose 

77 to be a very small number. Moreover, this will only introduce negligible conservativeness in 

the evaluation of coverage probabilities of random intervals if we choose r] to be sufficiently small 
(e.g., r? = 10-i5)_ 

To see the practical importance of AMCA in our statistical problems, consider the construction 
of a random interval with lower limit ^{9,n) and upper limit '^(6,n) such that Pr{^(0,n) < 
9 < "^{0,11)1 9} > 1-6, or equivalently, C{9) < 6 for any 9 G [9,9], where C{9) = Pr{^(0, n) > 
9 \ 9} + Fi{'^{d,n) < 9 \ 9} and [9,9] is a subset of 9. For our statistical problems, C{9) is 
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dependent on the coverage tuning parameter By choosing small enough ^, it is possible to 
ensure C{9) < 6 for any 6 E [6, 9]. To avoid unnecessary conservativeness, it is desirable to obtain 
C as large as possible such that C{9) < 6 for any 6 G \0_,6]. This can be accomplished by a 
computational approach. Clearly, an essential step is to determine, for a given value of C, whether 
C{9) < 6 holds for any 9 G [9,9]. Here, C{9) is defined as the complementary probability of 
coverage. To reduce computational complexity, C{9) can be replaced by its upper bound derived 
from the consecutive-decision variable bounding method proposed in Section 13. 2i 

In the case that is a discrete set, special care needs for d to ensure that a and b are numbers 
in Q. The backward AMCA can be easily modified as forward AMCA. 

3.4 Interval Bounding 

By virtue of statements (III) and (IV) of Theorem [3l we have 

Fr{^{d, n)>9ov'^{d,n)<9\9}> Pr{6 < ^{6, n) | a} + Pr{a > "^{6, n) | b}, (2) 
Pr{^(0, n) > 9 or {d,n) < 9 \ 9} < Pr{a < ^{6, n) | 6} + Fr{b > ^(0, n) | a} (3) 

for any 9 £ [a,b] provided that 

{a < ^{9, n)} C{d> b}, {b > ^(0, n)} C {0 < a}. (4) 

For many problems, if interval [a, b] is narrow enough, then, condition ^ can be satisfied and 
the upper and lower bounds of Pr{^(0, n) > 9 or '^{9,n) < 9 \ 9} in ^ and ([3]) can be used to 
determine whether Pr{^(0,n) > 9 or {0,n) < 9 \ 9} < 5 for any 9 G [a,b]. This suggests an 
alternative approach for constructing random intervals to guarantee prescribed confidence level 
for any 9 G [9, 9], where [9, 9] is a subset of parameter space G. The basis idea is as follows: 

(i) Construct sampling scheme such that the probabilities Pr{9 < ^{0, n) | 9} and Pr{^ > 
{6, n) I 9} can be adjusted by (. 

(ii) Partition [0, 9] as small subintervals [o, b] such that ([2|) and ([3]) can be used to determine 
whether Pr{if (0, n) > 9 or (d,n) < 9 \ 9} < 6 for any 9 G [a, 6]. 

3.5 Recursive Computation 

As will be seen in the sequel, for most multistage sampling plans with deterministic sample sizes 
n'i,n2,--- ,ns for estimating parameters of discrete variables, the probabilistic terms involving 

0, n or Oi, n£ can usually be expressed as a summation of terms Pr{A'j G ,Jti, i = 1, • • • i = 

1, • • • , s, where Kg = Xi and Jti is a subset of integers. The calculation of such terms can 
be performed by virtue of the following recursive relationship: 

Pr{J^, G ^, i = 1, • • • Ke+i = h+i} 
= ^ FriKiGJ^i, i = !,■■■ ,i- I; K£ = ke}Pr{K£+i-K£ = k£+i-ki}, (5) 
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where the computation of probabihty Prji^T^+i — Ki = ki^i — kf\ depends on specific estimation 
problems. For estimating a binomial parameter p with deterministic sample sizes ni, n2, • • • , n^, 
we have 

Pr{ir,+i -K, = ke+i - ke} = ~ l')p''+^-'''{l - 

For estimating a Poisson parameter A with deterministic sample sizes ni , n2 , • • • , , we have 

pr^ ^ , [(n,+i - ?i,)A]^^+i-^^ exp(-(n^+i - n,)A) 

PrjA^+i - Ki = ki+i -ki} = — . 

[ke+i - ki)\ 

For estimating the proportion, p, of a finite population using multistage sampling schemes de- 
scribed in Section 12. 6| we have 

/ pN-ki \ / N-pN-ni+ke \ 

Vt{Ki+i -Ke = ki+i - k^} = , (6) 

where the sample sizes are deterministic numbers ni,n2,--- ,ns. It should be noted that such 
idea of recursive computation can be applied to general multistage sampling plans with random 
sample sizes ni,n2,-- - ,ns. Moreover, the domain truncation technique described in the next 
subsection can be used to significantly reduce computation. 

3.6 Domain Truncation 

The bounding methods described in the previous subsection reduce the computational problem 
of designing a multistage sampling scheme to the evaluation of low-dimensional summation or 
integration. Despite the reduction of dimensionality, the associated computational complexity is 
still high because the domain of summation or integration is large. The truncation techniques 
recently established in [7] have the power to considerably simplify the computation by reducing 
the domain of summation or integration to a much smaller subset. The following result, quoted 
from [7], shows that the truncation can be done with controllable error. 

Theorem 9 Let ai, bi,Ui, Vi, ai, /3i, i = 1, - ■ ■ ,m be real numbers. Suppose that Pr{Zi < Ui} < ai 
and PT{Zi > Vi} < f3i for i = ,m. Then, P' < Prjcj < Zi < bi, i = I,-- - ,m} < 

P' + YlT^ii'^i + ft)' where P' = Pr{a^ 1^ Zi < b'-, i = 1, • • • ,m} with a\ = max{ai,nj} and 
b[ = min{6j, Uj} for i = 1, • • • , m. 

As an example of using the truncation technique, consider probabilistic terms like Vi{'W{6, n) G 
3i} involved in a multistage sampling scheme. If k^ and k^ can be found such that Pr{^ < Og < 
> 1-7 for ^ = !,••• ,s, then 

Fi{W{d, n)e^}-7]< PT{W(e, n) e^, kg<ee<ke, ^ = 1, • • • , s} < Pr{W{d, n) e ^}. 

For most multistage sampling plans for estimating parameters of discrete variables, the probability 
Pr{W{0, n) G Oi < 9i < 9i, i = 1, - ■ ■ ,s} can be evaluated recursively as described in Section 
[331 
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3.7 Triangular Partition 

As can be seen from the preceding discussion, by means of the double-decision-variable method, 
the design of multistage sampling schemes may be reduced to the evaluation of probabilities of 
the form Pr{(U,V) £ where U and V are independent random variables, and = {{u,v) : 
a < u < b, c < V < d, e < u + V < f} is a two-dimensional domain. It should be noted that 
such a domain can be fairly complicated. It can be an empty set or a polygon with 3 to 6 sides. 
Therefore, it is important to develop a systematic method for computing Pi{{U,V) € S^}. For 
this purpose, we have 

Theorem 10 Let a < b, c < d and e < f. Let e = max{e,a + c}, / = min{/, 6 + d}, u = 
max{a,e — d}, u = min{6, / — c}, y_ = max{c, e — 6} and v = minjd, / — a}. Then, for any 
independent random variables U and V, 

Pr{(f/, y) G ^} = Pr{n <U <u} Fr{v <V<v} 

- Pr{U < u, V <v, U + V> I}- Pr{U > u, V > v, U + V <e}. 



The goal of using Theorem [10] is to separate variables and thus reduce computation. As can be 
seen from Theorem llOl random variables U and V have been separated in the product and thus the 
dimension of the corresponding computation is reduced to one. The last two terms on the left side 
of equality are probabilities that (U, V) is included in rectangled triangles. The idea of separating 
variables can be repeatedly used by partitioning rectangled triangles as smaller rectangles and 
rectangled triangles. Specifically, if U and V are discrete random variables assuming integer 
values, we have 



Pt{U >i, V>i, U + V <k) ^ Pr <{ i < L/ < 



k + i-j 



PT{j<V< 



k — i + j 





'k - i + j' 


1 +Pr|[/ > i, V> 




2 



for integers i, j and k such that i + j < k; and 
Pt{U <i, V <j, U + V >k} = Pr ' ^ + ^^^ 



<U <i} Pr 



k — i + j 



U + V <k 



<V<j 



(7) 



<i, V < 


k — i + j 




2 



, U + V > k} +Pi {U < 



k + i — j 



, V<j, U + V>k 



(8) 



for integers i, j and k such that i + j > k. It is seen that the terms in ([7]) and ^ correspond to 
probabilities that {U, V) is included in rectangled triangles. Hence, the above method of triangular 
partition can be repeatedly applied. For the sake of efficiency, we can save the probabilities that U 
and V are respectively included in the intervals corresponding to the rectangular sides of a parent 
triangle, then when partitioning this triangle, it suffices to compute the probabilities that U and 
V are included in the intervals corresponding to two orthogonal sides of the smaller rectangle. 
The probabilities that U and V are included in the intervals corresponding to the rectangular 
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sides of the smaller triangles can be readily obtained from the results of the smaller rectangle and 
the record of the probabilities for the parent triangle. This trick can be repeatedly used to save 
computation. 

Since a crucial step in designing a sampling scheme is to compare the coverage probability 
with a prescribed level of confidence, it is useful to compute upper and lower bounds of the 
probabilities that U and V are covered by a triangular domain. As the triangular partition 
goes on, the rectangled triangles become smaller and smaller. Clearly, the upper bounds of the 
probabilities that {U, V) is included in rectangled triangles can be obtained by inequalities 

Pr{J7 > i, V>j, U + V <k}< Pr{i <U <k-j} Pr{j <V <k-i}, 

Pr{C/ < i, y < j, U + V >k} < Pr{fc -j<U<i} Vi{k -i<V <]}. 

Of course, the lower bounds can be taken as 0. As the triangular partition goes on, the rectangled 
triangles become smaller and smaller and accordingly such bounds becomes tighter. To avoid the 
exponential growth of the number of rectangled triangles, we can split the rectangled triangle 
with the largest gap between upper and lower bounds in every triangular partition. 

3.8 Interval Splitting 

In the design of sampling schemes and other applications, it is a frequently-used routine to evaluate 
the probability that a random variable is bounded in an interval. Note that, for most basic random 
variables, the probability mass (or density) functions /(.) possess nice concavity or convexity 
properties. In many cases, we can readily compute inflexion points which can be used to partition 
the interval as subintervals such that /(.) is either convex or concave in each subinterval. By 
virtue of concavity or convexity, we can calculate the upper and lower bounds of the probability 
that the random variable is included in a subinterval. The overall upper and lower bounds of 
the probability that the random variable is included in the initial interval can be obtained by 
summing up the upper and lower bounds for all subintervals respectively. The gap between the 
overall upper and lower bounds can be reduced by repeatedly partitioning the subinterval with 
the largest gap of upper and lower bounds. This strategy is referred to as interval splitting in this 
paper. 

For a discrete random variable with probability mass function f{k), we can apply the following 
result to compute upper and lower bounds of Y^k=a /(^) subinterval [a, h]. 

Theorem 11 Let a < b be two integers. Define Va ~ ^^frr'^ ~ ^^m)^ ^ ^'a.b = 77x7 o-nd j ~ 




Define a{i)^{i + l- a) 1 + ii^iilfcL^ and l3{i) = (b^i) 1 + 



2 



The following statements hold true: 

(I): If f{k + 1) - f{k) < f{k) - f{k -l)fora<k<b, then 



(9) 



k=a 
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for a < i < b. The minimum gap between the lower and upper bounds is achieved at i such that 

ljj < i < \j^ ■ 

(II): If f{k + 1) - f{k) > f{k) - f{k -l)fora<k<b, then 



(6-a + l)[/(a) + /(6)] 



> J]/(A:)>a«/(a) + /3«/(6) 



k=a 



for a < i < b. The minimum gap between the lower and upper bounds is achieved at i such that 

LiJ < i < Til • 

See Appendix O for a proof. For a continuous random variable with probability density 
function f{x), we can apply the following result to compute upper and lower bounds of f{x)dx 
over subinterval [a,b]. 

Theorem 12 Suppose f[x) is differentiable over interval [a,b]. The following statements hold 
true: 

(I): If fix) is concave over [a,b], then [/(°)+/WKfa-a) < flf{x)dx < +A{t), where 



_ f(b)-f{a) 



(b-tY 



(II): If fix) is convex over [a,b], then \fi'')+f(bm-a) _ ^^^^ < J^f(^x)dx < [/W+/W](b-a) _ 
The minimum of Ait) is achieved at t = . 



See Appendix iHl for a proof. 



3.9 Factorial Evaluation 

In the evaluation of the coverage probability of a sampling scheme, a frequent routine is the 
computation of the logarithm of the factorial of an integer. To reduce computational complexity, 
we can develop a table of ln(n!) and store it in computer for repeated use. Such a table can be 
readily made by the recursive relationship ln((n + 1)!) = ln(n + 1) + ln(n!). Modern computers 
can easily support a table of ln(n!) of size in the order of 10'' to 10^, which suffices most needs of 
our computation. Another method to calculate ln(n!) is to use the following double-sized bounds: 

ln(V2^n") -n + ^ - ^ < ln(n!) < ln( ^2^ n") - n + ^ " ^ + 
for all n > 1. A proof for such bounds can be available in pages 481-482 of |2U] . 



4 Estimation of Binomial Parameters 

Let X be a Bernoulli random variable with distribution Pr{X = 1} = 1 — Pr{X = 0} = p G (0, 1). 
In this section, we shall consider the multistage estimation of binomial parameter p, in the general 
framework proposed in Section [27H based on i.i.d. random samples Xi,X2, - ■ • of X. 
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To describe our estimation methods, we shall introduce the following notations, which will be 
used throughout this section. 

Define = X]^^ Xi and Pe = ^ for £ = !,••• , s, where is the number of samples available 
at the ^-th stage. Specially, if the sample sizes are deterministic numbers ni < n2 < ■ ■ ■ < rig, 
then ni = ni for £ = 1, • • • , s. As described in Section 12.11 the stopping rule is that sampling is 
continued until Di = 1 for some H. G {!,••• , s}, where is the decision variable for the £-th 
stage. Let p = — where n is the sample size when the sampling is terminated. Clearly, 
p = Pi and n = n;, where I is the index of stage when the sampling is terminated. As mentioned 
before, the number of stage, s, can be a finite number or infinity. 

In the development of our multistage sampling schemes, we need to use the following proba- 
bility inequalities related to bounded variables. 

Lemma 1 Let X.„ = '"^^ — -, where Xi, • • • , X„ are i.i.d. random variables such that < Xi < 
1 and K[Xi] = /-f G (0, 1) for i = 1, • • • , n. Then, 

Pr{Xn>z} < exp{n^B{z,fi)) (10) 

< exp {n^{z, fi)) (11) 

for any z € (/U, 1). Similarly, 

Pr{X„<z} < exp(n^B (2,Ai)) (12) 

< exp {n^{z, fi)) (13) 

for any z G (0, /i). 

Inequalities ([TO]) and ([12]) are classical results established by Hoeffding in 1963 (see, [27]). 
Inequalities ([TT]) and are recent results due to Massart [2U]. In this paper, ([TU|) and ([T^ 
are referred to as Hoeffding's inequalities. Similarly, (jlip and (jl3p are referred to as Mas- 
sart's inequalities. If Xi are i.i.d. Bernoulli random variables, then it can be shown that 
exp{n^-Q{z , fi)) = inf(>o e~*^ E[e*"'^"], which implies that ([TOj) and ([T2]) are actually Chernoff' 
bounds in the special case. 

4.1 Control of Absolute Error 

In this subsection, we shall propose multistage sampling schemes for estimating p with an absolute 
error criterion. Specifically, for margin of absolute error e G (0, |), we want to design a multistage 
sampling scheme such that the estimator p satisfies the requirement that Pr{|p— p| < e \ p} > 1 — 5 
for any p G (0, 1). 

4.1.1 Stopping Rules from CDFs, Chernoff Bounds and Massart's Inequality 

To construct an estimator satisfying an absolute error criterion with a prescribed confidence level, 
we propose three types of multistage sampling schemes with different stopping rules as follows. 
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Stopping Rule (i): For i = 1, • • • , s, decision variable assumes value 1 if Fg^ {Pi,Pi + < 
C(5, Gp^ (Pe^Pi ^ < C*^; and assumes value otherwise. 

Stopping Rule (ii): For £ = 1, • • • ,s, decision variable De assumes value 1 if ^b{\ ~ ~ 
p^l, i — |i — p^l + e) < ^^^^p-] and assumes value otherwise. 

Stopping Rule (iii): For £ = !,••• , s, decision variable Di assumes value 1 if (jp^ — i| — > 



1 



4 ^ 21n(C<5) 



and assumes value otherwise. 



Stopping rule (i) is derived by virtue of the CDFs of p^. Stopping rule (ii) is derived by 
virtue of Chernoff bounds of the CDFs of p^. Stopping rule (iii) is derived by virtue of Massart's 
inequality for the CDFs of p^. 

For the above three types of multistage sampling schemes, we have the following results. 

Then, 



2e2 



Theorem 13 Suppose that the sample size at the s-th stage is no less than 

s 

Pi-{p <p-e\p}<J2 Mp <Pi-s, Di^l\p}< sC6, 

1=1 

s 

Pr{p >p + e\ p]<Y^ Pr{p >Pi + e,Di = \\p}< sQS 

1=1 

and Pr{|p — p\<e\p}>\ — 2sC,5 for any p G (0, 1). 
See App endix II . 1 1 for a proof. 

For stopping rules derived from CDFs or Chernoff bounds, we can choose the smallest sample 
sizes and the largest sample sizes based on the criteria proposed in S ection [27T] such that ui > ^^(^^^Ij 

In i 

and n-s > Specifically, the sample sizes rii < 712 < • • • < can be be chosen as the ascending 
arrangement of all distinct elements of 



2^2 



(14) 



where r is the maximum integer such that ^ '-^ > j^^^^^, i.e., Cr~i > ■ In a sim- 

ilar manner, for stopping rules derived from Massart's inequality, the sample sizes ni < n2 < 
• • • < Us can be defined as (jl4p with r chosen as the maximum integer such that ^ ^^-^ '•^ > 

\ 9 ) ^P"' l-^-' ^^-1 - 9 • 

For above sampling methods of choosing sample sizes, we have Pr{|p — p\ <e|p}>l — 5 for 
any p € (0, 1) if ^ < where r is independent of 6. Hence, we can determine a value of C, as 
large as possible such that Pr{|p — p\ < e | p} > 1 — 5 by virtue of the computational machinery 
described in Section [3l 

To evaluate the coverage probability associated with the stopping rule derived from Chernoff 
bounds, we need to express events {D^ = i}-, i = 0, 1 in terms of K(^. This can be accomplished 
by using the following results. 
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Theorem 14 Let z* be the unique solution of equation In = , w!^ - with respect 



to z £ (i — e, i). Let he a sample size smaller than -^—t-S-I^t-^ ■ Let z he the unique solution 



z{l-z-e) (z+e)(l-z-e) 

0/ equation ^-q^z^z + e) = — w^^/i respect to z £ [0, z*). iei z 6e t/ie unique solution of 
equation ^^iz, z + e) = —^p- with respect to z £ (z*, 1 — e). Then, {Di = 0} = {n£Z < Ki < 
niz] U {n^(l -z)<Ki< ne{l - z)}. 

See Appendix 11.21 for a proof. 
4.1.2 Asymptotic Stopping Rule 

It should be noted that, for a small e, we can simplify, by using Taylor's series expansion formula 
ln(l + x) = X — ^ + o(x^), the sampling schemes described in Section 14.1.11 as follows: 

(i) The sequence of sample sizes ni,-- - ,ns is defined as the ascending arrangement of all 
distinct elements of | [^— ^-^jr-^ : £ = 1, - ■ ■ , rj, where r is the maximum integer such that Cr-i > 
2e. 

otherwise. 

For such a simplified sampling scheme, we have 

S ST 

Y,PT{\pe-p\>e,De = l} < ^Pr{|p, - p| > e} < ^Pr - p| > e} 

e=i 1=1 1=1 

T 

< ^2e-2"^^' (15) 
e=i 

-2ni£2 



(ii) The decision variables are defined such that Di = 1 if n£ > — — and Di 



< 2re"^"i^ < 2rexp ( -2eln— ) , (16) 



where (jl5p is due to the Chernoff bound. As can be seen from (|16p . the last bound is independent 
of p and can be made smaller than 5 if ^ is sufficiently small. This establishes the claim and it 
follows that Pr {|p — < e \ p} > 1 — 5 for any p £ (0, 1) if C is sufficiently small. 

4.1.3 Asymptotic Analysis of Sampling Schemes 

In this subsection, we shall focus on the asymptotic analysis of multistage sampling schemes. 
Throughout this subsection, we assume that the multistage sampling schemes follow stopping 
rules derived from Chernoff bounds as described in Section [4.1.11 Moreover, we assume that the 
sample sizes ni, • • • , are chosen as the ascending arrangement of all distinct elements of the set 
defined by p^. 

With regard to the tightness of the DDV bound, we have 
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Theorem 15 Let ^ be a subset of real numbers. Define 

s s 

P = Y^ G -D^-i = 0, = 1}, £ = 1 - MPi i Di_i = 0, De 



Then, P < Pr{p G ^} < P and liiiie^o | Pr{p £ ^} - P\ = linie^o | Pr{p G -l^} - £| 
(0,1). 



!}• 

/or any 



See Appendix 11.31 for a proof. 

For p > 0, d> 0, < u <1, define 



^{p, V, d) 



1 

2^ 



exp 



2^2 



2cos^ 



+ 



27r- 



exp 



d2 



2cos^ 



with cj)p — arctan(y/p), 0l = arctan ( i^/l + 1 + ^ j and = arctan ( 



to the asymptotic performance of the samphng scheme, we have 



Theorem 16 Let Naip.s) — ,i — . i 1 1 — rr^- Let Mdp.e) he the minimum, sample number 

n such that Pr{| „^ — ^ — p| < £ | p} > 1 — /o'^ ^ fixed-size sampling procedure. Let jp be the 
maximum integer j such that Cj > 4p(l — p). Let = |, d = ,/21n-i and = . Let 



d4> 



With regard 



4p(l-p) 



'"^Y3-T ^ 1 ior Kp = 1, Jp > and pp = Kp — 1 otherwise. The following statements hold true: 



PP ~ 4p(l-p) 

("/;.■ Pr |l < linisup^_^o JT^^ < 1 + Pp} = 1- Specially, Pr |lime-yo a/^^: 



3 J- — 1 if Hp 



> 1. 



r^^>li-^-o:^ = fe) xhm, 



Efnl 



hm 



>0 A^^' 



where 



E[nl 



p if i^p > ^1 

7Va(p, e) H _|_ p^f^iyd) otherwise 



E[n 



and 1 < lim,^o 

(^///j.- IfKp > 1, thenlims^o^T^{\p-p\ < e} = 2^ (d^/7t^)-l > 2$ (d)-l > 1-2(6. Otherwise, 
2$ (d) - 1 > lim£_,o Pr{|p - p\ < e} = 1 + $(d) - $(i/d) - ^{pp, L',d) > 3<l> (d) - 2 > 1 - 3(5. 



See Appendix 11.41 for a proof. 



4.2 Control of Relative Error 

In this section, we shall focus on the design of multistage sampling schemes for estimating the 
binomial parameter p with a relative error criterion. Specifically, for e G (0,1), we wish to 
construct a multistage sampling scheme and its associated estimator p for p such that Pr{|p— p| < 
I p} > 1 — (5 for any p G (0, 1). 



29 



4.2.1 Multistage Inverse Sampling 



In this subsection, we shall develop multistage sampling schemes, of which the number of stages, 
s, is a finite number. Let 71 < 72 < • • • < 7s be a sequence of positive integers. The number, 7^, 
is referred to as the threshold of sample sum of the i-th. stage. For £ = 1, • • • , s, let = where 
n£ is the minimum number of samples such that Yl?=i -^i ~ 7^- described in Section [2.H the 
stopping rule is that sampling is continued until £>£ = 1 for some i S {1, • • • , s}, where Di is 
the decision variable for the ^-th stage. Define estimator p = — where n is the sample size 
when the sampling is terminated. 

The rationale for choosing p as an estimator for p can be illustrated by the following results. 

Theorem 17 Suppose that inf£>o is greater than 1. Then E[p— p] and'E\p—p\'' , k = 1,2, ■ ■ ■ 
tend to as the minimum threshold of sample sum tends to infinity. 



See Appendix 11.51 for a proof. 

By virtue of the CDFs of p£, we propose a class of multistage sampling schemes as follows. 

Theorem 18 Suppose that, fori = 1, • • • ,s, decision variable assumes values 1 if Fp^{p^, < 
(S, Gp^{p£, j^) < (S; and assumes otherwise. Suppose that the threshold of sample sum for the 



s-th stage is equal to 



(l+£)ln(C^) 
e-(l+£)ln(l+e) 



. Then, 



Pr \p > I p] < ^Pr{p, < (1 - e)p, = 1 \ p} < 8(6, 

^ ^ ' i=i 

Pr Ip < I p\ < ^Pr{p^ > (1 + e)p, = 1 \ p} < 8(6 



(17) 
(18) 



for any p £ (0, 1). Moreover, Pr| < e | p| > 1 — S for any p G (0, 1) provided that ( is 



sufficiently small to guarantee 1 — 5p(7s — 1, + Spi^js — 1, < 6 and 



(1 + e + Vl + 4£ + £2) 1 
4^2 + 2 



1 + e 



ln(l + s) 





p-p 




{ 






p 





for any p £ [p* , 1), where p* G (0, Zg^i) denotes the unique number satisfying 



1 - 5p 7, - 1, 



7s 



1 + e 



5p f 7s - li Y3~) + y^exp(7£^i(z£,p*)) = 6 
^ ^ ^ £=1 



with Z£ = min{z G Ip^ : Fp^{z, j^) > (^''^ Gp^{z, j^) > C*^}; where I^^ represents the support 
of Pi, for 1 = 1,- ■■ ,s. 
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See Appendix 11.61 for a proof. Based on the criteria proposed in Section 12.11 the thresholds 
of sample sum 71 < 72 < • • • < 7s can be chosen as the ascending arrangement of all distinct 
elements of 

"Cr-i (l + e)ln(C<5) 



f = I,--- 



1 ' f ' 



(19) 



e- (l + e)ln(l + e) 

Cr-i (i+e)\n(CS) > ■ ^ ^ > . 

e-(l+e)ln(l+e) - RT+iJ' - ^ ' 

By virtue of Chernoff bounds of the CDFs of p^, we propose a class of multistage sampling 
schemes as follows. 



where r is the maximum integer such that i„(i+,y - i-^., ^r-i - ^ - ^-^^^-^^^^^^^^y 



Theorem 19 Suppose that, fori = 1, • • • , s, decision variable Di assumes values 1 if^i{pf, j^) < 
^"^^"^^ ; and assumes otherwise. Suppose that the threshold of sample sum for the s-th stage is 
equal to \-Jl±^}^^] . Then, 



Ft{p> 



P 



1-e 



P 



< MPi < (1 - ^)P, De = l\p}< sC5, 



(=1 



Pr !p < \p\ <Y^ Pr{p^ > (1 + £)P, De = l\ p} < 3(6 



(20) 
(21) 



for any p G (0, 1). Moreover, Pr| < £ | p| > 1 — 6 for any p G (0, 1) provided that ^ is 



sufficiently small to guarantee 1 — S'p(7s — 1, + S'p(7s — 1, j^) < 6 and 



HCS) < 
p-p 



Pr 



P 



(!+£ + Vl+4£ + £2) 1 

4^2 + 2 

<e\p\ >l-6 



l + e 



ln(l 



(22) 



for any p £ [p* , 1), where p* G (0, Zg-i) denotes the unique number satisfying 

1 - Sp (^7, - 1, + Sp (^7, - 1, + exp{j,^iize,p*)) = S 

where G (0, 1) is the unique number such that Jl\ [zg, -^^^ = '"^^"^^ for £ = 1, - ■ ■ ,s — 1. 



See Appendix 11.71 for a proof. Based on the criteria proposed in Section 12. H the thresholds 
of sample sum 71 < 72 < • • • < 7s can be chosen as the ascending arrangement of all distinct 
elements of the set defined by ([T9|) . 

It should be noted that both zi and p* can be readily computed by a bisection search method 
due to the monotonicity of the function ./#i(., .). 

By virtue of Massart's inequality for the CDFs of p^, we propose a class of multistage sampling 
schemes as follows. 
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Theorem 20 Suppose that, for 



1 



3+e 



: I,-- - ,s, decision variable De assumes values 1 if pp > 
(z+e)'^\n(cs) ' '^'^^ assumes otherwise. Suppose the threshold of sample sum for the s-th 



stage is equal to 



2(l+£)(3+e) ] 1 
P 



Then, 



Pr < p > 



1-e 



P 



< MPi < (1 - D^ = l\p}< sC5, 



Pr |p < ^Yl ^^^P<^ ^ (1 + ^)P^ De = l\ p} < sCS 

for any p E (0, 1). Moreover, Pr| < e | p| > 1 — S for any p G (0, 1) provided that ( is 
sufficiently small to guarantee 1 — S'p(7s — 1, -f^) + S'p(7s — 1, < 5 and 



HCS) < 



Pr 



P-P 



P 



<s\p\ >l-6 



1 + e 



- ln(l + e) 



for any p G [p* , 1), where p* G (0, 2:^-1) denotes the unique number satisfying 

s-l 



1 - 5p 7. 



1, 



Is 



with ze = l + ^^ ^ 



1 + e 

3+? ^ 2(3+e)^ ln(C<5) ■/'^'^ 



+ 5p ( 7. - 1, 

1 



7s 



1-e 
s-l. 



+ E 

£=1 



exp 



Z£ 



^{ze,p*) = 6 



See Appendix 11.81 for a proof. Based on the criteria proposed in Section 12.11 the thresholds 
of sample sum 71 < 72 < • • • < 7s can be chosen as the ascending arrangement of all distinct 
elements of 



2a- 



1 



1 1 



- + II I - + - 1 In — 



1 



where r is the maximum integer such that 2Cr-i + ( £ + 5) ^ — ^^9^^^ A' i-®-' C'^-i ^ 



2£ 



3(l+e) • 

It should be noted that {De = i} can be expressed in terms of n^. Specially, we have 
Do = 0, Ds = l and {De = 0} = {n^ > ^} for £ = 1, • • • , s - 1. 

To apply the truncation techniques of [7] to reduce computation, we can make use of the 
bounds in Lemma [55] and a bisection search to truncate the domains of n£_i and ne to much 
smaller sets. Since ne — ne^i can be viewed as the number of binomial trials to come up with 
7£ — 7£_i occurrences of successes, we have that — ne-i is independent of ne-i. Hence, the 
technique of triangular partition described in Section \37l\ can be used by identifying n^_i as U 
and ne — n^-i as V respectively. The computation can be reduced to computing the following 
types of probabilities: 

Pr{n < n,_, <v\p} = ± (^;_-_\) (j^)"" (1 " p)^ 



Pr{n < — ne^i < v \ p} = 



n — 1 \ / p 
je - le-i -ij \^-p 



7f-7f-l 
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where u and v are integers. 



From the definition of the sampHng scheme, it can be seen that the probabihties that p is 
greater or smaher than certain values can be expressed in terms of probabihties of the form 
Prjrij G Nj, i = 1, - ■ ■ ,i}, 1 < i < s, where Ni, • • • ,Ns are subsets of natural numbers. Such 
probabilities can be computed by using the recursive relationship 

Pr{ni e Ni, i = 1, • • • n^+i = ni+i} 
= ^ Pr{nj e N, , i = 1, • • • , ^ - 1; ne = ne} Pr{n^+i - = n^+i - ng} 



= ^ Pr{nj e Nj, i = 1, 1; = X 



for ^ = !,••• ,s-l. 



n-i+i ~ne-l 



(1 



With regard to the average sample number, we have 
Theorem 21 For any p G (0, 1], E[n] = ^ with E[j] = 71 + E£i(7^+i - le) Pi'{^ > 
See Appendix II. 91 for a proof. 

4.2.2 Asymptotic Stopping Rule 

We would like to remark that, for a small e, we can simplify, by using Taylor's series expansion 
formula ln(l + x) = x — ^ + o(x^), the multistage inverse sampling schemes described in Section 
14.2.11 as follows: 

(i) The sequence of thresholds 71 , • • • , 7s is defined as the ascending arrangement of all distinct 



elements of | 



2Cx-« In- 



1, • • • , t|, where r is the maximum integer such that Cr-i > 

^ ; and D, 



2- 



(ii) The decision variables are defined such that £>£ = 1 if 7£ > 
otherwise. 



(l-Pe) 2ln^ 



For such a simplified sampling scheme, we have 

S ST 

^Pi{\p£-p\>ep,De = l} < {\pf, - p\ > ep} < {\p£ - p\ > ep} 



i=i 



< 2r exp ( 71 



1 + e 



ln(l + e) 



1 + e 



ln(l + e) 



(23) 
(24) 



where (j23p is due to Corollary of [8]. As can be seen from ()24p . the last bound is independent 
of p and can be made smaller than 5 if <^ is sufficiently small. This establishes the claim and it 
follows that Pr {|p — p| < £p \ p} > 1 — 5 for any p € (0, 1) if C is sufficiently small. 
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4.2.3 Noninverse Multistage Sampling 

In Sections 14.2.11 and 14.2.2^ we have proposed a multistage inverse sampling plan for estimating a 
binomial parameter, p, with relative precision. In some situations, the cost of sampling operation 
may be high since samples are obtained one by one when inverse sampling is involved. In view of 
this fact, it is desirable to develop multistage estimation methods without using inverse sampling. 

In contrast to the multistage inverse sampling schemes described in Sections 14.2.11 and 14.2.21 
our noninverse multistage sampling schemes have infinitely many stages and deterministic sample 
sizes ni < 712 < ^3 < • • • . Moreover, the confidence parameter for the ^-th stage, 6i, is dependent 
on £ such that Si = 6 for 1 < i < t and 6e = 52"^"^ for £ > t, where r is a positive integer. 

By virtue of the CDFs of p^, we propose a class of multistage sampling schemes as follows. 



< 



2(t+1)- 



Theorem 22 Suppose that, fori = 1, 2, • • • , decision variable assumes values 1 if Fp^{p^, 
(Se, Gp^{pf, j^) < C,5i; and assumes otherwise. The following statements hold true. 

(I) : Pr{n < oo} = 1 provided that inf^>o > 1. 

(II) : E[n] < oo provided that 1 < inf£>o < sup^^g ^^^7" < oo. 

(III) : Pr I £^ <e\p'^>l-5foranype (0, 1) provided that ( < 

(IV) : LetO < T] < (6 and e* ^ T+l+l^-^^^^pi]. Then, Ft {\p - p\ > ep} < 6 for any p e (0,p*), 

where p* is a number such that < p* < Zi, £ = 1, ■ ■ ■ ,i* and that ex.p{n£^B{ze, p*)) < 6 — rj 

with Zi = min{z G Ip^ : Fp^{z, j^) > (6i orGp^{z, j^) > C6i}, where Ip^ represents the support 
of Pi, for i = 1,2, • • • . Moreover, 

Pr jfe < « < r I a| < Pr |p < ^ I p| < I + Pr |a < Z < r I 6 

Pr (a > l<£* \ b\ <PiIp> | pi < - + Pr (ft > I < £* \ a 

[ 1 — £ J 1^ 1 — eJ2 1^ 1 — £ 

for any p £ [a, b], where a and b are numbers such that 0<6<(l + e)a<l. 

(V) : Let the sample sizes of the multistage sampling scheme be a sequence Ui = |"m7^~^] , £ = 
1,2,---, where 7>1 + ^>1. Let < e < ^, < r] < I and c = sH^^. Let k be 
an integer such that k > max {r, ^ In In ^) + 1, r + ^ + and .J^b{vp, i^) < 
Then, E[n] < e + m + " '^^) ^^i^ > 

By virtue of Chernoff bounds of the CDFs of p^, we propose a class of multistage sampling 
schemes as follows. 

Theorem 23 Suppose that, for£ = 1, 2, • • • , decision variable Di assumes values 1 i/^B(P£, j^) < 
^^i^^- and assumes otherwise. The following statements hold true. 



(I) : Pr{n < oo} = 1 provided that inf^>o ^^^j^ > 1. 

(II) : E[n] < oo provided that 1 < inf^>o ^ sup^^g < 

(III) : Pr < e \ p^ > 1 - S for any p e (0,1) provided that ( < 2(7^' 
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(IV) : LetO <r] < (6 ande = T+1+ • Then, Ft {\p - p\ > ep} < 6 for any p e {0,p*), 
where p* is a number such that < p* < ze, £ ~ t, ■ ■ ■ ,£* and that J2e=i ^wi^e^Bi^e, p*)) < 5 — 7] 
with zn satisfying (^zi, ^[^^ = ^^^^e''' /^'^ ^ ~ 2, • • • . Moreover, 

Pr < « < n a| < Pr |p < ^ I p| < I + Pr |a < Z < r I , 

Pr la > Z < r I 6 1 < Pr \p > | pi < - + Pr (fe > Z < T | a 

[ 1 — e J [ 1 — £J2 [ 1— £ 

for any p G [a, b], where a and b are numbers such that 0<6<(l+e)a<l. 

(V) : Let the sample sizes of the multistage sampling scheme be a sequence = ["7717^"^] , I = 
1,2,---, where 7>1 + ^>1. Let Q < e < \, Q < rj < I and c = ^^i^^. Let k be 
an integer such that k > max {r, In (J- In ^) + 1, r + ^ + and ^Bim j^) < 
Then, E[n] < e + ni + E^=i(?^^+i - ^e) Pr{Z > £}. 

See Appendix 11.101 for a proof. 

By virtue of Massart's inequality for the CDFs of p^, we propose a class of multistage sampling 
schemes as follows. 

Theorem 24 Suppose that, for i = 1,2,---, decision variable assumes values 1 if Pi > 
2(3+^)'a\n(^j^g)"^9f^|;a / md assumes otherwise. The following statements hold true. 



(I) : Pr{n < 00} = 1 provided that inf^>o > 1- 

(II) : E[n] < 00 provided that 1 < inf£>o < sup^>o ^^^7" < ^jo. 

(III) : Pr I < e I p| > 1 - 5 /or any p G (0, 1) provided that C < jiT+Tj' 



(IV) : LetO <r] <C6 and£* = T+l+ • Then, Fr {\p - p\ > ep} < 6 for any p e {0,p*), 

where p* is a number such that < p* < z(, £ ~ t, - ■ ■ ,£* and that Yl^i=i exp(n^^(z£,p*)) < 6 — rj 
with zi = for i = l,2,--- . Moreover, 

Pr|5<^, Z<r|a|<Pr|p<^|p|<|+Pr|a<^, I < £* \ b 

Pr |a > i < I ^1 < Pr |p > I p| < | + Pr |& > ^ < ^* I « 

/or any p G [a, 6], where a and b are numbers such that 0<6<(l + e)a<l. 

(V) : Let the sample sizes of the multistage sampling scheme be a sequence ni = ["7717^"^] , £ = 
1,2,---, where 7 > 1 + ^ > 1. Let < e < ^, < i] < 1 and c = ^^iy^. Let k be 
an integer such that k > niax|T, i^ln(^ln^) +1, t + + and ^(?/p, j^) < 
Then, E[n] < e + ni + E^=i(^^+i - Pr{i > i}. 

4.2.4 Asymptotic Analysis of Multistage Inverse Sampling Schemes 

In this subsection, we shall focus on the asymptotic analysis of multistage inverse sampling 
schemes. Throughout this subsection, we assume that the multistage inverse sampling schemes 
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follow stopping rules derived from Chernoff bounds as described in Section 14.2. 1[ Moreover, we 
assume that the thresholds of sample sum 71 , • • ■ , 7s are chosen as the ascending arrangement of 
all distinct elements of the set defined by (^T9\i . 

With regard to the tightness of the double-decision-variable method, we have 

Theorem 25 Let ^ be a subset of real numbers. Define 

s s 

P = J2 P^iPf ^ =0, De = l}, Z = 1 - ^ MPi i ^, D,_i =0,1), = 1}. 

e=i 1=1 

Then, P < Pr{p £ ^} <P and liuie^o \ Pr{p £ -P\ = limg^^o I Pr{p £ ^} - P\ =0 for any 
pG (0,1). 



See Appendix II. 11 1 for a proof. 

Recall that I is the index of stage when the sampling is terminated. Define 7 = 7;- Then, 
7 = X^iLi -^i- With regard to the asymptotic performance of the sampling scheme, we have 

Theorem 26 Let j{p,e) — ^'^p^'^p ^ • -^^^ J^liPi^) be the minimum sample number n such that 
P'^il — - — p\ < ep \ p} > 1 — (5 for a fixed-size sampling procedure. Let jp be the maximum 
integer j such that Cj > 1 — p. Let v — d — ^2 In ^ and Kp = Let pp = — 1 if Kp = 1 

and Pp = tip — 1 otherwise. The following statements hold true: 

(I) : Pr |l < limsup^^o < 1 + Ppj = 1. Specially, Pr |lim.^o = Kpj = 1 if Kp > 1. 

(II) : lim,^o = (4^)' X lim,^o fgy, where 

EKl I Hp if Hp > 1, 

hm — = < 

6^0 7(p, £) otherwise 

and 1 < lim£_,o -^j^ < 1 + Pp- 

(III) : If Hp > 1, then lim^^oPrllp - p\ < ep} = 2<P {d^) - 1 > 2$ (d) - 1 > 1 - 2Q5. 
Otherwise, 2$ (d) - 1 > lim^^o Pr{|p -p\ < ep} = 1 + ^(d) - ^{i^d) - "^{pp, v, d) > 3$ ((i) - 2 > 
1 - 3C-5. 

See Appendix II. 121 



4.2.5 Asymptotic Analysis of Noninverse Multistage Sampling Schemes 

In this subsection, we shall focus on the asymptotic analysis of the noninverse multistage sampling 
schemes which follow stopping rules derived from Chernoff bounds of CDFs of Pi as described in 
Theorem [231 

We assume that the sample sizes ni , 712 , • • • are chosen as the ascending arrangement of all 
distinct elements of the set 

Gr-l HC5) 
^Bip*.^) 



(25) 
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with p* S (0, 1), where r is the maximum integer such that '^^ / ^"^f/^ > i-rnfr, i.e., Cr-i > 

ln(l+£) ■ 

With regard to the asymptotic performance of the samphng scheme, we have 

Theorem 27 LetAfr{p,e) = y Let J\ff{p, e) be the minimum sample number n such that 

Pr{| ^'^^ — ' ^ p\ < \ p} > ^ ^ fof 0, fixed-size sampling procedure. Let jp be the maximum 
integer j such that Cj > r[p), where r{p) ~ . Let v = |^^^, d = ^2 In and Kp = Let 

Pp = ^p^y — 1 if Kp = 1 and pp = Kp — 1 otherwise. For p G {p*, 1), the following statements hold 
true: 

(I): Pr |l < limsup^^^^o jj^^ < 1 + Ppj = 1- Specially, Pr |linie_^o jr^^ = Kpj = 1 if Kp > 1. 
(W^ l™-o ^ = (^l)' X li--o where 

lim = { " '-^"^ ^ ^' 

e^o 7V;(p, e) I X _^ pp^{vd) otherwise 

and 1 < lim£_,o < 1 + Pp- 

f///;.- // Kp > 1, i/ien lime_,oPr{|p - p| < ep} = 2^' (d^) - 1 > 2$ (d) - 1 > 1 - 2(5. 
Otherwise, 2$ (d) - 1 > Ume_^o Pr{|p -p\ <ep} = 1 + - ^>(z^d) - ^(pp, i/, d) > 3$ (d) - 2 > 
1 - 3C-5. 




See Appendix 11.131 for a proof. 
4.3 Control of Absolute and Relative Errors 

In this section, we shall focus on the design of multistage sampling schemes for estimating the 
binomial parameter p with a mixed error criterion. Specifically, for < < 1 and < < 1, 
we wish to construct a multistage sampling scheme and its associated estimator p for p such 
that Pr{|p — p\ < Sa, \p — 'p\ < ^rP I p} > 1 — (5 for any p G (0, 1). This is equivalent to 
the construction of a random interval with lower limit and upper limit (p) such that 

Pr{^(p) < p < ^{p) \ p} > I — 6 fov any p G (0, 1), where ^(.) and ^{.) are functions such 
that ^(-z) = min{z — £„, j^-} and (z) = max{z + e^, for z G [0, 1]. In the sequel, we 

shall propose multistage sampling schemes such that the number of stages, s, is finite and that 
the sample sizes are deterministic numbers ni < n2 < • • • < n^. 



4.3.1 Stopping Rules from CDFs and Chernoff Bounds 

To construct an estimator satisfying a mixed criterion in terms of absolute and relative errors 
with a prescribed confidence level, we have developed two types of multistage sampling schemes 
with different stopping rules as follows. 
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Stopping Rule (i): For i = I,-- - ,s, decision variable Di assumes value 1 if F^^{pi,,'^ [p^)) < 
C6, Gp^{pg,^{pg)) < (S; and assumes value otherwise. 

Stopping Rule (ii): For £=!,••• , s, decision variable Di assumes value 1 if 
max{^B(Pf, -^(Pf)), ■-^BiPe,'^ (Pe))} < ^'^n^ and assumes value otherwise. 

Stopping rule (i) is derived by virtue of the CDFs of p^. Stopping rule (ii) is derived by virtue 
of Chernoff bounds of the CDFs of p^. For both types of multistage sampling schemes described 
above, we have the following results. 

Theorem 28 Let and be positive numbers such that < < || and ^^^2le < 1- 



ln(C-5) 



. Then, 



Suppose that the sample size for the s-th stage is no less than 

s 

Vt{p < ^{p) I p} < ^ Pr{p < ^{pi,), Di = l\p] < sC5, 

£=1 

s 

Pr{p > ^(p) I p] < ^Pr{p > '^{Pi), Di = l\p] < sCS 
e=i 

and Pr{|p — p\ < Ea or \p — p\ < £rP | > 1 — 2sCS for any p £ (0, 1). 

See Appendix II. 141 for a proof. Based on the criteria proposed in Section [2. !( the sample sizes 
ni < n2 < ■ ■ ■ < Us can be chosen as the ascending arrangement of all distinct elements of the set 



Cr-e HCS) 



-1,---,T , (26) 



where r is the maximum integer such that > i^i^iH^) i i-e-, Cr-i > -—ji(Y^^Y^' 

For such a choice of sample sizes, as a result of Theorem [28l we have that Pr{|p — p\ < 
£a or |p — p| < £rP \ p} > I — S for any p G (0, 1) provided that C < 27- 

For computing the coverage probability associated with a multistage sampling scheme following 
a stopping rule derived from Chernoff bounds, events {D^ = i}, i = 0,1 need to be expressed as 
events involving only Kg. This can be accomplished by using the following results. 

Theorem 29 Let p* = ^. For £ = 1, ■ ■ ■ , s - I, {Di = 0} = {.£Yi{Pt,-^{Pt)) > U 



BiPij'^iPi)) > } and the following statements hold true: 
(I) {./#b(P£, .^(Pf)) > ^'^nP } = < ^'e < ng z+} where z+ is the unique solution of 

equation J^B{z,jf^) ~ ^'^fP ^^^^ respect to z £ {p* + £a,l], and z~ is the unique solution of 
equation ^b{z, z — Sa) = ^^v^ ^^^^ respect to z £ i£a,P* + ^a)- 

(W 

U0<Kg<ne z~} for n, < j^^^, 

|^b(p„'^(p,)) > ^1 = <^ {ng z+ <K,<m z-} for ^^^^ <n,< 



(p*-eo,P*) ' 
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where is the unique solution of equation ^b(z, jz^) = ^'^n^ with respect to z £ {p* — £a, l — Ej-), 
and z^ is the unique solution of equation ^b{z, z + Sa) = '"^^"^^ with respect to z G [0,p* — Ea)- 



See Appendix II . 1 51 for a proof. 



4.3.2 Stopping Rule from Massart's Inequality 

By virtue of Massart's inequality of the CDFs of p^, we can construct a multistage sampling 
scheme such that its associated estimator for p satisfies the mixed criterion. Such a sampling 
scheme and its properties are described by the following theorem. 



Theorem 30 Let Ea and Er be positive numbers such that < < | and 



6ea 



< Er < I- Suppose 



the sample size for the s-th stage is no less than 



for i — |£q 



D, 



1 ^ 2, 



21n(C<5) 



<Pe< 



. Define 



6(l-E^)(3-e^) ln(Ci5) 



1 , neel ^ ^ 6(l+£^)(3+£^) In(Cg) 
4 21n(C5) ^ ^ 2(3+e^)2 1n(C5)-9nie2 , 



1 else 



for I = 1, 



, s. Then, 

s 

Pt{p < ^(p) \ p} <Y^ Fi{p < ^{pe 

s 

Pv{p >'^{p)\p}< ^Pr{p > '^(pf 



Dp 



Dp 



l\p]< sC5, 
l\p} < sC5 



and Pr{|p — p\ < Ea or \p — p\ < ErP \ p} > 1 — 2s(6 for any p € (0, 1) 



See Appendix II. 161 for a proof. Based on the criteria proposed in Section [2.11 the sample sizes 
ni < n2 < ■ ■ ■ < Us can be chosen as the ascending arrangement of all distinct elements of 



2Cr 



1 



1 1 



1 Mi 1 



where r is the maximum integer such that 2Cr~e (7 — 7 — ^) (7" + ^)l^^ — ^^^T^l'^ A 



9er 



i.e., 1 > 3 y- gj . For such a choice of sample sizes, as a result of Theorem 1301 we 

have that Pr{|p — p\ < Ea or |p — j5| < ErP | > 1 — 5 for any p G (0, 1) provided that C < ^• 



4.3.3 Asymptotic Stopping Rule 



It should be noted that, for small Ea and Er, we can simplify, by using Taylor's series expansion 
formula ln(l + x) = x — ^ + o{x'^), the sampling schemes described in Section [4.3. II as follows: 
(i) The sequence of sample sizes ni,--- ,ns is defined as the ascending arrangement of all 



2Cr-i ^ - ^ 



distinct elements of | 



integer such that Cr~i > ( 7 — 7- 



,t| with Ea < where r is the maximum 
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Pl{l-Pf) 21n^ 

(ii) The decision variables are defined such that Df = 1 if m > r^—/ — - \Si ; and Dp = 

otherwise. 

For such a simpHfied samphng scheme, we have 

s s 

Pr - p| > max{ea, Erp}, De = l} < ^ Pr - p\ > max{ea, e,.p}} 

£=1 

T 



e=i 



< 



^2exp 



— + ea,— 



< 2r exp ( ni^B [ — + £a,— 



(27) 
(28) 



where (j27p is due to Theorem 1 of [6]. As can be seen from (|28p . the last bound is independent 
of p and can be made smaller than 5 if ^ is sufficiently small. This establishes the claim and it 



follows that Pr I |p 



p\ <£a or 



I p| > 1 — (5 for any p £ (0, 1) if C is sufficiently small. 



4.3.4 Asymptotic Analysis of Sampling Schemes 

In this subsection, we shall focus on the asymptotic analysis of multistage inverse sampling 
schemes. Throughout this subsection, we assume that the multistage sampling schemes follow 
stopping rules derived from Chernoff bounds as described in Section [4.3.11 Moreover, we assume 
that the sample sizes ni, • • • , are chosen as the ascending arrangement of all distinct elements 
of the set defined by (f26]) . 

With regard to the tightness of the double-decision- variable method, we have 

Theorem 31 Let M he a subset of real numbers. Define 

s s 

P = Y, e ^, De^i = 0, = 1}, £ = 1 - I] MPi i = 0, - 1}. 



Then, P < Pr{p G ^} < P and lime^^^o |Pr{p G ^} - -P| = hnie^^^o |Pr{p G - P| = for 
any p G (0, 1), where the limits are taken under the constraint that ^ is fixed. 

See Appendix 11.171 for a proof. 

With regard to the asymptotic performance of the sampling scheme as and Er tend to 0, 
we have 

Theorem 32 Let Mi(j>,Ea,Er) be the minimum sample number n such that 



Pr 



< £a or 



<erp\p} >1-CS 
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ln(C.5) 



for a fixed-size sampling procedure. Let A/'m(p, £a, £r 

rfe} andp = max{p + ea, j^}- Define p* = d=J2\n^, 



, where p = min{p 



r{p) 



forp€iO,p% 



2 p(l~p)(l-2p*) 
3p*(l-p*)(l-2p) 



2 p-p* 

3 1-p 



/orp G {0,p*], 
forp G (p*, 1). 



Lei Kp = where jp is the maximum integer j such that Cj > r{p). Let pp = 'rf^ ^ 1 ^/ 
Kp = 1, jp > and Pp = Kp — 1 otherwise. The following statements hold true under the condition 
that is fixed. 



(I): Pr |l < limsup^^^o A/„(p"a,£.) ^ 1 + Pp} = 1- Specially, Pr |lime„^o 



Kp| = I if 



Kp > I. 



(II)-- lim... 



>0 Aff(p,e„e,) 



I X lim, 



IE[n] 

«->0 Afm(p,£a,er)' 



where 



E[n] _ I ^//^p > 1, 

Ea^o 7Vm(p, ffa, Er) 1 1 + pp^[vd) otherwise 



lim 



and 1 < lim,„^o M^pZ^sr) <^ + Pp- 

(III): If Kp > 1, i/ien lime^_j.o Pr{|p-p| < or < Erp} = 2^ [d^Jl^^ - 1 > 2$ (d) - 1 > 

1 - 2C5. Otherwise, 2$ (d) - 1 > linie^^o Pr{|p - p| < or \p - p\ < Erp] = 1 + - - 
^'(pp, I/, d) > 3«> (d) - 2 > 1 - 



See Appendix II. 181 for a proof. 



5 Estimation of Bounded- Variable Means 

In the preceding discussion, we have been focusing on the estimation of binomial parameters. 
Actually, some of the ideas can be generalized to the estimation of means of random variables 
bounded in interval [0, 1]. Formally, let X G [0, 1] be a random variable with expectation p = E[X]. 
We can estimate /i based on i.i.d. random samples Xi,X2,--- of X by virtue of multistage 
sampling schemes. 

5.1 Control of Absolute Error 

To estimate the mean of the bounded variable X G [0, 1] with an absolute error criterion, we have 
multistage sampling schemes described by the following theorems. 

Theorem 33 Let < e < ^. Let ui < n2 < ■ ■ ■ < Ug he a sequence of sample sizes such 
that Hs > ^jf- Define Ji^ = '^^ — - for £ = 1, • • • ,s. Suppose that sampling is continued until 

-^b{\ ~ ~ /^^I'l ~ 1^ ~ A^l + ^) ^ ^i^'^^i^s)' I^^fi''^^ P- = ^^ir~^' where n is the sample size 
when the sampling is terminated. Then, Pr — /i| < e} > 1 — 5. 
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See Appendix II. 191 for a proof. 



Theorem 34 Let < e < ^. Let ni < n2 < ■ ■ ■ < rig be a sequence of sample sizes such 

that Hs > ^jf- Define Jl^ = '^^ — - for £ = 1, • • • , s. Suppose that sampling is continued until 

ilP-e ~ 5I ~ t) - i ^ 2 in(2" /i5) some £ G {1, • • • , s}. Define Ji = — ^, where n zs i/ie sample 
size when the sampling is terminated. Then, Pr — ;u| < e} > 1 — (5. 

See Appendix 11.201 for a proof. 
5.2 Control of Relative Error 

To estimate the mean of the bounded variable X E [0, 1] with a relative precision, we have 
multistage inverse sampling schemes described by the following theorems. 

Theorem 35 Let < e < 1. Let 71 < 72 < • • • < 7s 6e a sequence of real numbers such that 
7i > ^ and 7., > (i+e)^in(i+|).^e • For I = 1, - ■ ■ ,s, define Ji^ = where is the minimum sample 
number such that Yltli-^i — 7^- Suppose that sampling is continued until ^b{-^, ni(i+e) ) — 
± In (I) and ^b(;^, < In (i) for some £ e {1, • • • , Define U = where I 

is the index of stage when the sampling is terminated. Then, Pr {|^ — /i| < e/i} >1 — 5. 

Theorem 36 Let < e < 1. Let 71 < 72 < • • • < 7s 6e a sequence of real numbers such that 
71 > ^ andjs > ~ _ For i = 1, • • • ,s, define Ji^ = where is the minimum sample 

number such that ^^^iXi > 7^. Suppose that sampling is continued until -^(^1^7^(14^) ^ 
^ In (I) and ^(^, < In (|) for some £ e {1, • • • , s}. Define = |' where I 

is the index of stage when the sampling is terminated. Then, Pr{|^ — /i| < e^} >1 — 5. 

In some situations, the cost of sampling operation may be high since samples are obtained one 
by one when inverse sampling is involved. In view of this fact, it is desirable to develop multistage 
estimation methods without using inverse sampling. In contrast to the multistage inverse sampling 
schemes described above, our noninverse multistage sampling schemes have infinitely many stages 
and deterministic sample sizes ni < n2 < < ■ ■ ■ . Moreover, the confidence parameter for the 
^-th stage, 61, is dependent on i such that 61 = 6 for 1 < £ < t and 6£ = 62'^~^ for £ > t, where 
T is a positive integer. As before, define p,£ = '^^ — - for £ = 1, 2, • • • . The stopping rule is that 
sampling is continued until D£ = 1 for some stage with index £. Define estimator p. = Jli, where 
I is the index of stage at which the sampling is terminated. We propose two types of multistage 
sampling schemes with different stopping rules as follows. 

Stopping Rule (i): For £ = 1,2, decision variable assumes value 1 if ^bIm^, < 
i^^^^; and assumes value otherwise. 

Stopping Rule (ii): For £ = 1,2, ■ ■ ■ , decision variable Di assumes value 1 if 

^ ^ 6{l + e){'S + e)HCSi) 
- 2(3 + e)2 IniCde) - 9nes^ ' 
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and assumes value otherwise. 



Stopping rule (i) is derived by virtue of Chernoff-HoefFding bounds of the CDFs of /i^. Stopping 
rule (ii) is derived by virtue of Massart's inequality of the CDFs of 

Theorem 37 For both types of multistage sampling schemes described above, the following state- 
ments hold true: 

(I) : Pr{n < 00} = 1 for any G (0, 1) provided that inff>o > 1. 

(II) : E[n] < 00 for any fi G (0, 1) provided that 1 < inf^^o ^ sup^^g < ^jo. 

(III) : Pr I ^ <e \ >1-S for any fie (0, 1) provided that ( < 2(7+T)- 



5.3 Control of Absolute and Relative Errors 

In this subsection, we consider the multistage estimation of the mean of the bounded variable 
with a mixed error criterion. Specifically, we wish to construct a multistage sampling scheme and 
its associated estimator /x for ^ = E[X] such that Pr{|/x — /i| < Ea, \fl — ^\ < er/u} > 1 — 6. In 
the special case that the variable X is bounded in interval [0, 1], our multistage sampling schemes 
and their properties are described by the following theorems. 

Theorem 38 Let < < H and 35!°!!^ < < 1- Let ni < n2 < ■ ■ ■ < Us be a sequence of 
sample sizes such that Us > ^^['11^/^ ) ■ Define p,^ = ^1 — ■^if'-e) — ~ ^a, 

and {fii) = max{ju£ + f^''^ ^ ~ i-^- Suppose that sampling is continued until 

ma,x{^B{flg, J2f{flg)), .^B(/i£; '^(m^))} < TTjl^l^)- Define fi = — -, where n is the sample size 
when the sampling is terminated. Then, Pr{|/i — lJ-\ < £a or\fl — fi\ < ErfJ-} > 1 — 6. 

See Appendix 11.211 for a proof. 

Theorem 39 Let < < | and < Cr < 1- Let ni < n2 < ■ ■ ■ < Ug be a sequence of sample 
sizes such that ns>2(^j- + \^ (7- - 7- - |) In (^) . Define p,^ = ^' and 



n for 1 - 2p _ / 1 1 "fgg <- M <- 6(l-6.)(3-e.)lnCT 
U iUl 2 3ta Y 1__2MC^ '^^ 2(3-er)2 1n(C<5)-9rif62 Oi 

Df=l 1 + 2p _ /l I ritel ' ^ 6(l+£^)(3+£^)ln(C6) 



2 ^ 3^<» V ■* 21n(C<5) ^ ^ 2(3+e^)2 1n(C5)-9n;62 , 

1 else 

for i = 1, - ■ ■ ,s. Suppose that sampling is continued until = 1 for some i £ {!,■ ■ ■ ,s}. Define 
fl = — ^, where n is i/ie sample size when the sampling is terminated. Then, Pr{|^ — /i| < 

Ea or \fl — IJ,\ < er/U} > 1 — 

See Appendix 11.221 for a proof. 

In the general case that X is a random variable bounded in [a, b], it is useful to estimate the 
mean /i = K[X] based on i.i.d. samples of X with a mixed criterion. For this purpose, we shall 
propose the following multistage estimation methods. 
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Theorem 40 Let > and < Sr < 1- Let ni < n2 < ■ ■ ■ < rig be a sequence of sample sizes 
such that ns > 1^ (^). Define = n] — f^e = + bir^P-it 

f^f = ^ ~ ^fl' N } , t^e = «+7 max <^ fi^ + e^, 



— * — a [ 1 + sgn(/2^)er J o - a [ 1 — sgn(/i£jer 

/or£ = !,••• , s. Suppose that sampling is continued until ^BiPe, f^) ^ ^^^4^ and ^Bifj'ii'Pe) < 
— In 2j /c"^ some ^ G {1, • • • , s}. Define fi = '"^ " ' , where n is i/ie sample size when the sampling 
is terminated. Then, Pr{|/i — ;u| < or |/i — /i| < er|/u|} >1 — 5. 

Theorem 41 iyei > and < < 1- ni < n2 < • • • < he a sequence of sample sizes 
suc/i i/iat Us > -^^^S^ In (^)- Define fl^ = — ^, /x^ = a + jj^pi, 

A*/' = ^+1 ~ ^fl' s } , fJ-e = a+T max <^ /^^ + e^, 



— * — a [ 1 + sgn(/i^)er J b — a [ 1 — sgn(/i£)er 

/or i = 1, - ■ ■ ,s. Suppose that sampling is continued until ^{Jl^, fi^) < In ^ and ^(/i^, /I^) < 
— In 2^ /or some £ ^ {!, - ■ ■ ,s}. Define = '"^ ' ' , where n is i/ie sample size when the sampling 
is terminated. Then, Pr{|/i. — /i| < or l/i - /i| < erVII > 1 - 5. 

5.4 Using the Link between Binomial and Bounded Variables 

Recently, Chen [lOj has discovered the following inherent connection between a binomial parameter 
and the mean of a bounded variable. 

Theorem 42 Let X be a random variable bounded in [0,1]. Let U a random variable uniformly 
distributed over [0,1]. Suppose X and U are independent. Then, E[X] = Pr{X > U}. 

To see why Theorem 1421 reveals a relationship between the mean of a bounded variable and a 
binomial parameter, we define 

1 for X>U, 
otherwise. 



Y 



Then, by Theorem Eg we have Pr{y = 1} = 1 - Pr{y = 0} = E[X]. This implies that Y 
is a Bernoulli random variable and E[X] is actually a binomial parameter. For a sequence of 
i.i.d. random samples Xi, X2, • • • of bounded variable X and a sequence of i.i.d. random samples 
UiiU2, - ■ ■ of uniform variable U such that that Xi is independent with Ui for all i, we can define 
a sequence of i.i.d. random samples Yi,Y2, - ■ ■ of Bernoulli random variable Y by 



1 i0TYi>Ui, 

otherwise. 



As a consequence, the techniques of estimating a binomial parameter can be useful for estimating 
the mean of a bounded variable. 
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6 Estimation of Poisson Parameters 



In this section, we shall consider the multistage estimation of the mean, A, of a Poisson random 
variable X based on its i.i.d. random samples Xi, X2, • • • . 

For I = 1, 2, • • • , define = Xi, = where ni is deterministic and stands for the 
sample size at the £-th stage. As described in the general structure of our multistage estimation 
framework, the stopping rule is that sampling is continued until Di = 1 for some i € {!,■ ■ ■ , s}. 
Define estimator X = Xi, where I is the index of stage at which the sampling is terminated. 
Clearly, the sample number at the completion of sampling is n = nj. 

6.1 Control of Absolute Error 

In this subsection, we shall focus on the design of multistage sampling schemes for estimating the 
Poisson parameter A with an absolute error criterion. Specifically, for e > 0, we wish to construct 
a multistage sampling scheme and its associated estimator A for A such that Pr{|A — A| < e | 
A} > 1 — 6 for any A G (0,oo). As will be seen below, our multistage sampling procedures 
have infinitely many stages and deterministic sample sizes ni < n2 < < ■ ■ ■ ■ Moreover, the 
confidence parameter for the ^-th stage, 6e, is dependent on i such that 5e = 5 for 1 < i < t and 
6£ = 62'^~^ for £ > T, where r is a positive integer. 

6.1.1 Stopping Rule from CDFs 

By virtue of the CDFs of A^, we propose a class of multistage sampling schemes as follows. 

Theorem 43 Suppose that, for i = 1, 2, • • • , decision variable assumes values 1 if F-^^{\i,Xi,+ 
e) < C^e, G^^{Xi,\i — e) < QSt; and assumes otherwise. The following statements hold true. 

(I) : Pr{n < 00} = 1 provided that inff>o > 1- 

(II) : E[n] < 00 provided that 1 < inf^>o < sup^>o < 00. 

(III) : Pr{|A -A|<£|A}>l-5 for any A > provided that ( < 2{7+T)- 

(IV) : LetO < T] < C6 and £* = T + 1+ \^-^^^^] ■ Then, Pr{|A - A| > e | A} < 5 for any A e 

(A, 00), where X is a number such that \> zi, £ = !,■■■,£* and that "^^^i exp(n£^p(2;£. A)) < 6—7] 
with Zi = min{z G I^^ : (z, z + e) > (6e or G^^ (z, z — e) > C^i}, where I-^^ represents the 

support of Xi, for £ = 1,2, ■ ■ ■ . Moreover, 

Pr !& < A - e, i < r I a| < Pr |a < A - £ I a| < I + Pr |a < A - e, l<£* \bj, 
Pr |a > A + £, i < r I < Pr |a > A + £ I a| < ^ + Pr !& > A + £, I < £* \ 

for any A G [a, b], where a and b are numbers such that < b < a + e. 

(V) : Let the sample sizes of the multistage sampling scheme be a sequence ui = |'m7^~^] , £ = 
1, 2, • • • , where 7 > 1 + ^ > 1- L&t e>0, 0<77<1 and c = — ^p(^, A). Let k be an integer such 
that K > maxlr, .J-ln(^) +1, 1 In(^ln^) +1, r + ^ + ^^| and ^p(^,^+e) < ^Jii^. 

1 ' ln7 \crn/ '1117 V crn ce / ' 7— 1 In 2] r \ ^ ^ > j 

Then, E[n] < e + ni + ELi(f^^+i - ^i) Pr{l > £}. 
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6.1.2 Stopping Rule from ChernofF Bounds 

By virtue of Chernoff bounds of the CDFs of , we propose a class of multistage sampling schemes 
as follows. 

Theorem 44 Suppose that, for i = 1, 2, • • • , decision variable Di assumes values 1 if ^p{Xi, Xe + 
s) < '"n^*^*^ ^'^^ assumes otherwise. The following statements hold true. 

(I) : Pr{n < oo} = 1 provided that inf^>o ^^^j^ > 1. 

(II) : E[n] < oo provided that 1 < inf^>o ^ sup^^g < 

(III) : Pr{|A -A|<£|A}>l-5 for any A > provided that ( < 2(r+i) ■ 

(IV) : Let id <ij < C,5 and t = T + l+ [^^^^^1 • Then, Pr{| A - A| > e | A} < 5 for any X £ 

(A, oo), where X is a number such that X > zi, £ ^ t, ■ ■ ■ ,£* and that Yle=i exp(n^^p(z£, A)) < 6—7] 
with Ze satisfying ./#p {ze, ze + e) = ^^^^7^ for £ = 1,2, ■ ■ ■ . Moreover, 

Pr !& < A - e, i < r I a| < Pr |a < A - e I a| < I + Pr |a < A - e, I < £* \ 
Pr |a > A + e, i < T | foj < Pr |a > A + e | a| < | + Pr |6 > A + e, Z < T | a| 

for any X G [a, b], where a and b are numbers such that < b < a + e. 

(V) : Let the sample sizes of the multistage sampling scheme be a sequence = |'m7^~^] , I = 
1, 2, • • • , where 7 > 1 + ^ > 1- Let e>0, 0<t/<1 and c = — ^p(^, A). Let n be an integer such 
that K > maxlr, ^ln(^) +1, 1 In(^lni) +1, r + ^ + ^| and Jiv{-.-^e) < 

1 ' ln7 \cra) '1117 V crn ce / ' 7— 1 In 21 ^^r)^ r) ^ 

Then, E[n] < e + ni + - Pr{i > 

See Appendix IJ.ll for a proof. 



6.1.3 Asymptotic Analysis of Multistage Sampling Schemes 

In this subsection, we shall focus on the asymptotic analysis of the multistage sampling schemes 
which follow stopping rules derived from Chernoff bounds of CDFs of \i as described in Theorem 
SI 

Let A* > 0. We assume that the sample sizes ni,n2, • • • are chosen as the ascending arrange- 
ment of all distinct elements of the set 

:£=1,2,---|, (29) 

where r is the maximum integer such that |a*+£) — ^t-i ^ +^) _ With 

regard to the asymptotic performance of the sampling scheme, we have 

Theorem 45 Let Ma{\,s) — J^{\^\^e) ■ Let J\ff{X,e) be the minimum sample number n such that 
^""{1^ — ' — X\ < £ \ X} > 1 — C5 for a fixed-size sampling procedure. Let j\ be the largest integer j 
such that Cj > Let v = |(1 — p-), d ~ ^2 In^ and kx ~ x^ja • -^^^ Pa = x^ja-i ~ 1 ^f k-x = ^ 
and px = K\ — 1 otherwise. For A G (0, A*), the following statements hold true: 



^p(A*, A* +e) 
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(I) : Pr |l < limsup^_,(, j^lfey < 1 + Pa| = 1- Specially, Pr jlinie^o A/ip^ = ka| = 1 «/ > 1. 

(II) : lim,^o ^ = (47)' X lim,^o where 

lim ^["] - /'^^ '-^^^ > ^' 

£^oA/'a(A,e) ll + pA^'(j^d) otherwise 

and 1 < lime^o _v ! !(A!e) ^ 1 + Pa ■ 

("///;.■ //ka > 1, i/ienlime_,oPr{|A-A| < e} = 2<I> (d^)-! > 2$(d)-l > 1-2(^5. Otherwise, 
2$ (d) - 1 > lim£^oPr{|A - A| < e} = 1 + ^{d) - <^{yd) - ^{px,v,d) > 3$ (d) - 2 > 1 - 3C<5. 



See Appendix IJ.2I for a proof. 
6.2 Control of Relative Error 

In this subsection, we shall focus on the design of multistage sampling schemes for estimating 
the Poisson parameter A with a relative error criterion. Specifically, for e € (0, 1), we wish to 
construct a multistage sampling scheme and its associated estimator A for A such that Pr{| A — A| < 
eX \ X} > 1 — 5 for any A G (0,oo). As will be seen below, our multistage sampling procedures 
have infinitely many stages and deterministic sample sizes ni < n2 < < ■ ■ ■ ■ Moreover, the 
confidence parameter for the £-th stage, 6(_, is dependent on £ such that 5^ = 5 for 1 < I < t and 
5i = 52'^~^ for i > T, where r is a positive integer. 

6.2.1 Stopping Rule from CDFs 

By virtue of the CDFs of A^, we propose a class of multistage sampling schemes as follows. 

Theorem 46 Suppose that, fori = 1, 2, • • • , decision variable assumes values 1 ifF^^{Xe, j^) < 
(Se, G-^^{\i, < Q5i; and assumes otherwise. The following statements hold true. 
(I): Pr{n < 00} = 1 provided that inf^>o > 1- 



(II): E[n] < 00 provided that 1 < inf£>o < sup^>o m 



(III): Pr I < e I a| > 1 - 5 /or any A > provided that C, < 2(t+i) ■ 



(IV): Let <7] <C5 and£* = T + l+ i^^^^ . Then, Pr{|A - A| > eA | A} < for any A G 

(0, A), where X is a number such that < X < zi, £=!,■■■,£* and that Xl^^x e'Kp{n£^p{zi, X)) < 
5 — 7] with zn = min{z G /^^ : F-^^i^z, > C,bi or G-^^{z, j^) > C^e}> where I^^ represents the 
support of Xi, for i = 1,2, ■ ■ ■ . Moreover, 

Pi{b< i<r|aj><Pr<'A<^|Aj><^+Pr<'a< I < t \ b\ , 
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for any A G [o, b], where a and b are numbers such that < 6 < (1 + e)a. 

(V) : Pr{| A — A| > eA I A} < for any A E (A, oo), where X is a number such that X > zi and 
that 2exp(ni^p((l + e)X, A)) + exp(ni^p(2;i, A)) < 6. 

(VI) : Let the sample sizes of the multistage sampling scheme be a sequence = \^m'y^~^^^ , £ = 
1, 2, • • • , where 7 > 1 + ^ > 1- Let e>0, 0<r/<l and c = —^p{riX, A). Let k be an integer such 
that K > max(r, ^lii(^) + l, ^ln(^ln^)+l, t + ^ + ^-^\ and J^p{gX,^) < i^^. 

1 ' 1117 V cm / ' 1117 \ cm ce / ' 7 — 1 In 2 J V / ' l-\-s ^ 

Then, E[n] < e + ni + Yle=iine+i - ng) Pr{Z > £}. 

6.2.2 Stopping Rule from ChernofF Bounds 

By virtue of Chernoff bounds of the CDFs of A^, we propose a class of multistage sampling schemes 
as follows. 

Theorem 47 Suppose that, for I = 1,2, decision variable Di assumes values 1 if \e > 
'"n^"^'^ '-(i+^tin(i+') '" '^^^ assumes otherwise. The following statements hold true. 

(I) : Pr{n < 00} = 1 provided that inf^>o > 1- 

(II) : E[n] < 00 provided that 1 < inf^^o ^ sup^^g < 

(III) : Pr{|A - X\ < eX \ X} > I - 6 for any X > provided that ( < 2{7+Tj- 

(IV) : LetO<r] <C5 ande" = T + 1+ [^^^^^1 • Then, Pr{|A - A| > eA | A} < for any X £ 
(0, A), where X is a number such that < X < zg, £ = t, ■ ■ ■ , £* and that Yle=i 6xp(n£^p(z£, A)) < 
6 - V with z, = '^^^y ,_a+.lrn(i+ei /or ^ = 1, 2, • • • . Moreover, 

Vx{h< ^r^, l<£* \a} <Pi-{X< I A ^ < ^ + Pr <; a < l<£*\h^, 

Pr <; a > « < r I 6 j < Pr jA > | A j < ^ + Pr j5 > « < ^* | a| 

for any X G [o, b], where a and b are numbers such that < 6 < (1 + e)a. 

(V) : Pr{| A — A| > eA I A} < for any X G (A, 00), where X is a number such that X > zi and 
that 2exp(ni^p((l + e)X, A)) + exp(ni^p(2;i. A)) < 6. 

(VI) : Let the sample sizes of the multistage sampling scheme be a sequence = \m'y^~^^^ , £ = 
1, 2, • • • , where 7 > 1 + ^ > 1- Let e>0, 0<r7<l and c = —^p{rjX, A). Let k be an integer such 
that K > maxlr, ^lii(^) + l, 1 ln(^ln^)+l, r+^ + iif^| and J^p{t^X,^) < i^^. 
Then, E[n] < e + ni + E^=i(^^+i - ^e) > 

See Appendix IJ.3I for a proof. 

6.2.3 Asymptotic Analysis of Multistage Sampling Schemes 

In this subsection, we shall focus on the asymptotic analysis of the multistage sampling schemes 
which follow stopping rules derived from Chernoff bounds of CDFs of A^ as described in Theorem 
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Wl\ We assume that the sample sizes ni,n2, • • • are chosen as the ascendmg arrangement of ah 
distinct elements of the set 

''a-._MC^ :£=1,2,...S (30) 

with < A' < A", where r is the maximum integer such that '^^-i > ^lii^L^ 

.^p(a',^) - .4'p(a",^) 

Cr-i > ^ ^^^^ ^„ ■ With regard to the asymptotic performance of the sampling scheme, we 
have 

Theorem 48 Let Afr{X,e) = -s^rr^tr- Let M{{X,e) be the minimum sample number n such that 
P^'{| — A| < eA I A} > 1 — for a fixed-size sampling procedure. Let j\ be the largest integer j 



such that Cj > y . Let d = ^2 In ^ and n\ = jrCj^ . Let p\ = p-Cj^-i ^ 1 ^/ = 1 md p\ = k\ — 1 
otherwise. For A G (A', A"), the following statements hold true: 

(I) : Pr |l < limsup^^o N'X>^,e) < 1 + Pa} = 1- Specially, Pr |linie^o jf:^ = ka| = 1 «/ ka > 1- 

(II) : lim,^o ^ - (4;)' X lim.^o j^y where 



^.^ E[n] _ J «/ka>1, 
e^oA/;(A,e) + ^ otherwise 

and 1 < lime^o < 1 + PA- 

("///;.■ lime^^o Pr{| A - A| < eA} = 2$ [d^) - 1 > 2^> (d) - 1 > 1 - 2Q5. 

See Appendix IJ. 41 for a proof. 
6.3 Control of Absolute and Relative Errors 

In this section, we shall focus on the design of multistage sampling schemes for estimating Poisson 
parameter A with a mixed error criterion. Specifically, for > and < < Ij we wish to 
construct a multistage sampling scheme and its associated estimator A for A such that Pr{|A— A| < 
Ea, I A— A| < ErX | A} > 1 — (5 for any A G (0, oo). This is equivalent to the construction of a random 
interval with lower hmit -Sf (A) and upper limit ^{\) such that Pr{^(A) < A < '^(A) | A} > 1-5 
for any A G (0, oo), where .^{.) and ^(.) are functions such that »2'(z) = minjz — e^, and 
^{z) = max{z + Ea, jz:^} for z G [0,oo). In the sequel, we shall propose multistage sampling 
schemes such that the number of stages, s, is finite and that the sample sizes are deterministic 
numbers ni < n2 < • • • < n^. 

6.3.1 Stopping Rules from CDFs and Chernoff Bounds 

To estimate A with a mixed precision criterion, we propose two types of multistage sampling 
schemes with different stopping rules as follows. 
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Stopping Rule (i): For 1= I,-- - ,s, decision variable Di assumes value 1 if F^^{\i,'^ < 
(^S, G3^^(Af,^(Af)) < (S; and assumes value otherwise. 

Stopping Rule (ii): For i = I, - ■ ■ , s, decision variable Di assumes value 1 if 

max{.^p(A,, Jf(A,)), ^piXi,^ (Xi))} < i^^; 

and assumes value otherwise. 

Stopping rule (i) is derived by virtue of the CDFs of A^. Stopping rule (ii) is derived by virtue 
of Chernoff bounds of the CDFs of A^. For both types of multistage sampling schemes described 
above, we have the following results. 



Theorem 49 Suppose that the sample size for the s-th stage is no less than 
Then, 



Pr{A < ^(A) I A} < ^Pr{A < ^(Xi), = 1 | A} < sC5, 

s 

Pr{A > ^(A) I A} < J]]Pr{A > ^(Af), De = 1 \ X} < sCS 

i=\ 

for any A > 0. Moreover, Pr{|A — A| < or < Er \ X} > 1 - S for any A > provided that 

Pr{A < ^(A) I A} + Pr{A > (A) | A} < (5 for any A € (0, A], where X > is the unique number 
satisfying J2e=i exp(n£^p(A(l + e^), A)) = |. 

See Appendix IJ. 51 for a proof. Based on the criteria proposed in Section [2.11 the sample sizes 
ni < n2 < ■ ■ ■ < Us can be chosen as the ascending arrangement of all distinct elements of 



Cr-i HCS) 



:) 



£=1,---,T , (31) 



where r is the maximum integer such that F"". si '"^''ti > — —, i.e., Cr-i > P^er"*" "'er^ _ ppj. 

such a choice of sample sizes, as a result of Theorem 1491 we have that Pr{|A — A| < Sa or < 
I A} > 1 — (5 for any A > provided that C < 27- 

To evaluate the coverage probability associated with a multistage sampling scheme following 
a stopping rule derived from Chernoff bounds, we need to express {Di = i} in terms of Ki. For 
this purpose, the following result is useful. 

Theorem 50 Let X* = ^. Then, {D^ = 0} = {^p(Af , if (A^)) > ^^Mi } U {^p(A<,, ^(A^)) > isg^} 
for i = 1, - ■ ■ ,s — 1 and the following statements hold true: 

(I) {^p{Xe,J2f{Xe)) > ^-^^^p-} ~ {ni < Kg < ui z+} where is the unique solution of equation 
Jip{z, xfr^ ^ '"^"^"^^ with respect to z ^ (A* + Ea, oo), and z~ is the unique solution of equation 
^p{z,z - £a) = ^^^^ with respect to z € (£„, A* + Eq). 
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(II) 



.^P(A,, ^(A,)) > i^^l = {n, zt <K,<n, z"} for <n, < ^^ffitl.AM - 



n ^ ln(C5) 



where z^. is the unique solution of equation ^p(z, jz^) = ^^^^p- with respect to z €z (A* — oo), 
and z^ is the unique solution of equation ^p{z, z + Sa) = '"^^'^^ with respect to z £ [0, A* — 



Theorem [50] can be shown by a variation of the argument for Theorem 
6.3.2 Asymptotic Stopping Rule 

It should be noted that, for small Ea and e^, we can simplify, by using Taylor's series expansion 
formula ln(l + x) = x — ^ + o{x'^), the sampling schemes as described in Section [6.3.11 as follows: 

(i) The sequence of sample sizes ni,--- ,ns is defined as the ascending arrangement of all 
distinct elements of | Cr-i (^^^ ^ ■ ^ = ■ ■ where r is the maximum integer such that 

Cr-l > f . " 

(ii) The decision variables are defined such that Dp = 1 if > /„ , '•^ : and Di = 

max{£^, (srAf)^} 

otherwise. 

For such a simplified sampling scheme, we have 

s s 

y^Pr ||A£ - A| > max{ea,£rX}, De = l| < y^Pr||A^-A| > maxjeg, e^Ajj 



r 

- y^Pr||A^-A| > max{ga,g,.A}| 

< ^2exp(^n,^p(^^ + g„,^)) (32) 

< 2rexp ^ni^p + , 



(33) 

where (|32p is due to Theorem 1 of j9j. As can be seen from (I33p . the last bound is independent 
of A and can be made smaller than 5 if C is sufficiently small. This establishes the claim and it 
follows that Pr I A — A < £a or < | a| > 1 — 5 for any A € (0, oo) if is sufficiently small. 

6.3.3 Asymptotic Analysis of Multistage Sampling Schemes 

In this subsection, we shall focus on the asymptotic analysis of multistage inverse sampling 
schemes. Throughout this subsection, we assume that the multistage sampling schemes follow 
stopping rules derived from Chernoff bounds as described in Section [6.3.11 Moreover, we assume 
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that the sample sizes ni, • • • ,ns are chosen as the ascending arrangement of aU distinct elements 
of the set defined by (f3T]) . 

With regard to the tightness of the double-decision-variable method, we have 

Theorem 51 Let S?. he a subset of real numbers. Define 

s s 

P = J2 P""!^^ e = 0, = 1}, Z = 1 - ^ Pr{A, i = 0, = 1}. 

e=i i=\ 

Then, P < Pr{A G ^} < P and lime„^o I Pr{A e .'f } - P| = linij„->o I Pr{A G if} - P| = /or any 
A G (0,oo), where the limits are taken under the constraint that |2- is fixed. 

See Appendix IJ.6I for a proof. 

With regard to the asymptotic performance of the sampling scheme as and £r tend to 0, 
we have 

Theorem 52 Let J\fi{\,£a^£r) be the minimum sample number n such that 



- X 



< £n or 



- A 



<erX\X \ >l-CS 



for a fixed-size sampling procedure. Let Mm{X,ea,£r) ~ 



max{^p (A, A^), ^p(A,A)} 



, where A = min{A 



^a, '^^'^ ^ ~ ™ax{A + £a, t:^}- Define A* = d = ^2 In 

r(A) 



^ /or A G (0, A* 



A 



V = < 



1(1-^) /or A G(0, A* 







/or A G (A*,oo). 



r(A) 



- 1 



for A G (A*,oo) 

Let Kx = where jx is the maximum integer j such that Cj > r(A). Let p\ 

if Kx = 1, jx > and px = n\ — ^ otherwise. The following statements hold true under the 
condition that |^ is fixed. 

Pr |l < limsup^^^o _;v;„(A"„,e,o < 1 + Pa} = 1- Specially, Pr jlinie^^o A4,(A"„.e.) = ''^a} = 1 if 
Kx>l. ^ 

(II): lim,„^o M.^^ler) = Mrl^Lsr) ' "'^e'^e 



lim 



Efnl 



KA 



ifKx > 1, 



Ea^o AAmlA, £„, Er) 1 1 + pA*l'(i^t^) otherwise 

and 1 < lim,„_^o m^a"!,.,-) ^ ^ + ^a- 

(^/J/;.- //ka > 1, i/ienlim£„^oPr{|A-A| < or | A- A| < e,.A} = 2$ ((i^)-l > 2$((i)-l > 
1 - 2C(5. 

If Kx = l and A > A*, i/ien lim^^^o Pr{| A - A| < or | A - A| < A} = 2$ (d) - 1 > 1 - 2Q5. 
If Kx = I and A < A*, then 2$ (d) - 1 > lime^^oPr{|A - A| < £a or |A - A| < e^A} = 
1 + - ^{vd) - ^(pA, I', d) > 3$ (d) - 2 > 1 - 3C5. 



See Appendix [X3 for a proof. 
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7 Estimation of Finite Population Proportion 



In this section, we consider the problem of estimating the proportion of a finite population, 
which has been discussed in Section 12. 6[ We shall focus on multistage sampling schemes with 
deterministic sample sizes ni < n2 < • • • < ?is. Our methods are described in the sequel. 

Define Ke ~ J27=i P£ = "ff for ^ = 1, • • • , s. Suppose the stopping rule is that sampling 
without replacement is continued until Di = 1 for some i € {!,■ ■ ■ ,s}. Define p ~Pi, where I is 
the index of stage at which the sampling is terminated. 

By using various functions to define random intervals, we can unify the estimation problems 
associated with absolute, relative and mixed precision. Specifically, for estimating p with margin 
of absolute error e G (0,1), we have Pr{|p — p\ < e} = Pr{^{p) < p < ^(p)}, where ^(.) 
and ^ (.) are functions such that -Sf (z) = jj \N{z — e)] — jj and (z) = [N{z + e)J + for 
z S [0, 1]. For estimating p with margin of relative error e G (0, 1), we have Pr{|p — p\ < sp} = 
Pr{^(p) <p< ^(p)}, where and '^(.) are functions such that ^(z) = ^ \Nz/{l + e)]-j^ 
and ^{z) = lNz/{l — e)J + ;^ for z G [0, 1]. For estimating p with margin of absolute error 
Ea € (0, 1) and margin of relative error G (0, 1), we have Pr{|p — p\ < Sa or |p — p| < £rP} = 
Pr{^(p) < p < ^ (p)}) where ^(.) and '^(.) are functions such that 

1 



A^min 



-a; -I 

l+Er 



N max [ z + Sa 



1 



for z S [0,1]. Therefore, multistage estimation problems associated with absolute, relative and 
mixed precision can be cast as the general problem of constructing a random interval with lower 
limit ^{p) and upper limit »Sf(p) such that Pr{^(p) < p < ^(p)} > 1 — 6. For this purpose, 
making use of Theorems [2] and El we immediately obtain the following result. 

Corollary 3 Suppose the sample size of the s-th stage is no less than the minimum number n such 
thatl-SNik-l,n,^{^)) < (5 and SN{k,n,'^ {^)) <Q5forQ<k<n. For £ = 1, ■ ■ ■ , s, define 
Di such that Di assumes value 1 if 1 — S^iKt — ^,^i,-^iPe)) < C^^ SN{Ki,ni,'^ (p^)) < ^5; 
and assumes value otherwise. Then, 



Pi{p < if (p) \p}<Yl Pr{p < ^{pe), De = l\p}< <<5, 

s 

Pt{p > ^(p) I p} < ^Pr{p > 'W{Pi), Di = l\p} < sCS 
e=i 

and Pr{if (p) < p < ^{p) \ p} > 1 — 2sC,5 for any p e Q. 
Let 

nmin = 1 + max|7i : 1 - Sat ^fc - l,n,if (j^^ > or Sn {k,n,'^ (n)) > for < fc < n , 
"max = min (77. : 1 - 5jv ( fc - l,n,if ( - ) ) < and Sn (k,n,'^ ( -]] < (S for < k < n 
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Based on the criteria proposed in Section [2.11 the sample sizes ni < n2 < ■ ■ ■ < Ug can be chosen 
as the ascending arrangement of ah distinct elements of the set {{Cr-e "maxl ■ I < £ <t}, where r 
is the maximum integer such that Cr-i > """" . 



Now, define 

C{z,p,n,N) 



Try 



for z = 1, 

/Np\(N~Np\ (34) 

IF™)F 



^jf^ry^^jYp^zrjjw+TjiT] for z G {- : A; e Z, < A; < n} 



where p £ Q. In order to develop multistage sampling schemes with simple stopping boundaries, 
we have the following results. 

Corollary 4 Suppose the sample size of the s-th stage is no less than the minimum number n 
such that C{^,^{^),n,N) < (6 and C(^, ^(^), n, iV) < (6 for < k < n. For £ = I, ■ ■ ■ , s, 
define such that Di assumes value 1 if C{pi,^(pi),ni,N) < (6, C{p£,'^ (j)£),ni, N) < (6; 
and assumes value otherwise. Then, 



Pr{p < ^(p) \p}<Yl ^""^P ^ ^(P^)' Di = l\p}< sCd, 

s 

Pt{p > ^(p) I p} < ^Pr{p > '^{Pi), De = l\p} < 3(6 

and Pr{^(p) < p < ^{p) \ p} > 1 — 2s(6 for any p e Q. 

Corollary H] can be shown by using Theorems [21 [5] and the inequalities obtained by Chen [13] 
as follows: 

Pr 1 ^^=^^' < z I p| < C{z,p,n,N) for z £ : k £ Z, np < k < , (35) 

Pr 1 ^^=^^' > z I p| < C{z,p,n,N) for z e : k e Z, < k < np^ (36) 

where p G @. Since Y17=i -^i ^ hyper geometric distribution, the above inequalities ([35]) and 
([5U|) provide simple bounds for the tail probabilities of hypergeometric distribution, which are 
substaintially less conservative than Hoeffding's inequalities [27] . 

It is well known that, for a sampling without replacement with size n, to guarantee that the 
estimator p= ^'=^ ^' of the proportion p = satisfy {\p — p\ < e} > 1 — 6, it suffices to have 
n > p(i_p)+ff_'i^^2/^|^^ , or equivalents, Z^^^ (^ - l)p(l-p) < {N~l)e^ (see formula (1) in page 41 
of [38]). Therefore, for a very small margin of absolute error e, we can develop simple multistage 
sampling schemes based normal approximation as follows. 

To estimate the population proportion p € with margin of absolute error e G (0, 1), we can 
choose the sample sizes ni < n2 < • • • < as the ascending arrangement of all distinct elements 
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Zls(--l)M^-Pi)<{N-l)e' 



is satisfied at some stage with index i. Then, Pr {|p — j>| < e | p} > 1 — 5 for any p £ Q provided 
that the coverage tuning parameter C is sufficiently small. 

To estimate the population proportion p £ Q with margin of relative error e £ (0, 1), we can 
choose the sample sizes ni < n2 < • • • < as the ascending arrangement of all distinct elements 
of the set { \NCr-i] : i = 1, - • • , r}. The stopping rule is that sampling is continued until 



is satisfied at some stage with index £. Then, Pr {|p — p| < ep \ p} > 1 — 5 for any p £ Q provided 
that the coverage tuning parameter is sufficiently small. 

To estimate the population proportion p £ Q with margin of absolute error Sa £ (0, 1) and 
margin of relative error £r £ (0, 1), we can choose the sample sizes rii < n2 < • • • < as 
the ascending arrangement of all distinct elements of the set {[n^Cr- : i=l,--- ,r}, where 
n* = pw^_p^f^|]y'-i)e2 iz'^ with = < ^ • The stopping rule is that sampling is continued until 



is satisfied at some stage with index i. Then, Pr{ |p — p| < £a or |p — p| < £rP \ p} ^ 1 — 5 for any 
p £ Q provided that the coverage tuning parameter is sufficiently small. 

8 Estimation of Normal Mean 

Let X be a normal random variable of mean fi and variance o"^. In many situations, the variance 
(T^ is unknown and it is desirable to estimate fi with predetermined margin of error and confidence 
level based on a sequence of i.i.d. random samples Xi,X2, ■ • ■ of X. 

8.1 Control of Absolute Error 

For a priori e > 0, it is useful to construct an estimator /2 for ^ such that Pr{|/1 — fi\ < e} > 1 — 6 
for any fi £ (—00,00) and a £ (0, 00). 

8.1.1 New Structure of Multistage Sampling 

Our new multistage sampling method as follows. Define 




- - 1 - Pi) <{N-1) max{e^, {srPe)'} 




n 



Xn — 



n 
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for n = 2, 3, • • • , oo. Let s be a positive number. The sampling consists of s + 1 stages, of which 
the sample sizes for the first s sta ges ar e chosen as odd numbers rii = 2ki + 1, I = 1, - ■ ■ ,s with 
ki < k2 < ■ ■ ■ < kg. Define = \/-^^ for £ = 1, • • • , s. Let the coverage tuning parameter C, be 
a positive number less than ^. The stopping rule is as follows: 

If ni < {a-g Un-i^sY I ■, t — 1, - ■ ■ ,i — I and n, > (ff; tri,-i,c<5)^/£^ for some i G {1, • • • , s}, then 
the sampling is stopped at the i-th stage. Otherwise, \{as tn^-i^sY 1^'^'] ^ more samples of X 
needs to be taken after the s-th stage. The estimator of /i is defined as /i = — where n is 
the sample size when the sampling is terminated. 

It should be noted that, in the special case of s = 1, the above sampling scheme reduces to 
Stein's two-stage procedure [36] , 



Theorem 53 The following statements hold true. 

(I) Pr{|/i — /i| < e} > 1 — 2sC,5 for any and a. 

(II) lim,^oPr{|/i - ^i\< e] = l- 2(6. 

(III) E[n] < 

(IV) hmsup.^oIE [g] < (%^)^ where C = ( 



a Z 



2 



See App endix IK . II for a proof. 

As can be seen from statement (II) of Theorem I53| to ensure Pr{|/x — fi\ < e} > 1 — 6, it 
suffices to choose the coverage tuning parameter ( to be less than ^. However, such a choice is 
too conservative. To reduce sampling cost, it is possible to obtain a value of ^ much greater than 
^ by an exact computational approach. Such an approach is explored in the sequel. 

8.1.2 Exact Construction of Sampling Schemes 

To develop an exact computational approach for the determination of an appropriate value of 
coverage tuning parameter (, we need some preliminary results as follows. 

Theorem 54 Let 1 = k^ < ki < k2 < ■ ■ ■ be a sequence of positive integers. Let = zq < zi < 
Z2 < ■ ■ ■ be a sequence of positive numbers. Define h{0, 1) = 1 and 

h{i,l) = l, /i(£,m) = y ^^""''^Z^^",^"^"' \ kr<m<kr+i, r = 0,l,--- ,£-1 

^-^ (m — iy. 

1=1 ^ ' 

for I = 1, 2, • • • . Let Zi, Z2, • • • be i.i.d. exponential random variables with common mean unity. 
Then, 

Z,n > z, /or j = 1, • • • i = e-^' Yl ^(^'"^) 

m=l J m=l 

for i = 1,2, ■ ■ ■ . Moreover, the following statements hold true. 
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(I) 



rn—l 



2—1 I m— 1 

where Ai = [ai], Bi = \bi\ and 



i—l I m— 1 



Ar ar+l/2r-lxl 
-Br 6r+l-^2'— 1x1 



r+l 



r = 1,2,-- 



i?r Or+l-^2''-lxl 

Ar 6r.+l-^2'— 1x1 

where 12^-1x1 represents a column matrix with all 2"^^^ elements assuming value 1 

(W 



kt + 1 



Pr < flj < ^ < for j ^ 1, - ■ ■ ,1, ^ Z„ > 



m — 1 
^^-1 ( kj 



where E 
(III) 



^Pr^ Y,Zm>m,,forj = l,--- ,1+1 

i=l [m=l 

fe£+l-^2*-lxl 



^ Pr i ^ Zrr. > [F],,, /or J = 1, •••,£+ 1 



■i— 1 I ?n — 1 



and F 



'^+1-' 2^-1x1 



P^ S % < X! ^™ for j = !,■■■ ,t, ^ Z„ < &£+i 



m—l 



kt+i 



I 0.J < Zm < bj for j = 1, - ■■ ,i> - Pi <.aj < Z,n < bj for j = I, ■ ■ ■ J, ^ > be+i 



m—l 



771 — 1 



m—l 



For the purpose of computing appropriate coverage tuning parameter the fohowing results 
are useful. 

Theorem 55 Let the sample sizes of the sampling scheme be odd numbers ne = 2ke + 1, i = 
1, • • • , s, where 1 = ko < h < k2 < ■ ■ ■ < kg. Let bo = and bi = \''^]^^l'^^lyi for i = 1, • • • ,s. 
Define h{0, 1) = 1, h{£, 1) = 1, 

''^ h{r,i) (6,-6,)— 



h{i,m) = Y, 



i=l 



[m — i)\ 



-, kr < m < /Cr+li = 0, 1, 



and He{a) = e-'^ Y.t=i K^^ m) for £ = 1, ■ ■ ■ , s. Define c = j^f^, h*il) = I, 



^ > ^ {m-i)\ 



i=l 



kr < m < kr+i, r = 0, 1, • • • , s — 1 



57 



an. 



d H*{a, n) = e X]m=i h*{m) for n > rig. Then, the following statements hold true. 



1 - $ 



(I):Pr{\n-f^\>e} = 2Zn^y 
N : n > Us}. 

' He-i{a) - He{a) 



Pr{n = n}, where 5^ = {ug : 1 < i < s}L){n £ 



(II): Pr{n = n} = < 



for n = n£, 1 < £ < s, 
II*{a,n — 1) — II*{a,n) for n> rig 



where IIq{(t) = 1. 
(Ill): For any a G [a, h], 

Pr{|/i-M|>e}>2^ 

ne.s" 

Pr{|/i-Ml>£}<2^ 



1 - $ 



1 - $ 



P„ + 2 



1 - $ 



e^fm 



Sp ks - 1, 



(a in.-i,c<5)" 



where 



Pr. 



He^i{b) - Hi{a) forn = ne, 1 <£< s, 

II*{b, n — 1) — //*(a, n) for n> Ug 

Hi-i{a) - Hi,{h) forn = ni, 1 <£< s, 

H*{a,n — 1) — H*{b,n) forn>ns 



and m> Ug 
(IV): 



E[n] = m + - ne)He{a) + ^ H*{a,n) 



s-l 



< n 



where 7 



Us — I 



and m > max{ ^ , n^} . 



See Appendix IK. 21 for a proof. 

The coverage tuning process requires evaluation of the coverage probability Pr{|/x — /i| < e} 
for various values of a. To reduce the evaluation of coverage probability with respect to o" to a 
finite range of a, we have the following results. 

Theorem 56 Let the sample sizes of the sampling scheme be odd numbers = 2k^ + 1, £ = 
1, • • • ,s, where 1 < ki < k2 < ■ ■ ■ < kg. Suppose the coverage tuning parameter Q is a positive 
number less than ^ . Then, there exists a unique number a such that 



E 



I -Spike -I 



ne ki 
(ct tn^-ixs)^ 



(1 - 
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and that Pr{|/i — /i| > < S for a > a . Similarly, there exists a unique number a_ such that 



1 _ $ 



a 



s-2 

1=1 



1 - $ 



and that Pr{|/x — /Li| > e} < 5 /or cr < a. 



See Appendix IK. 31 for a proof. 



8.2 Control of Relative Error 



Sp {kg - 1, 



n-e kg e 



For a "priori e > 0, it is a frequent problem to construct an estimator /i for [i such that Pr{|/i — ;u| < 
^\l^W > 1 — (5 for any G (— oo,0) U (0,oo) and a G (0,oo). For this purpose, we would like to 
propose a new sampling method as follows. 



Theorem 57 Define Sg = 6 for 1 < i < t and 
For £ = 1, 2, • • • , let u« = ' and 



o-g = 



= 52'^ ^ for £ > T, where t is a positive integer. 
-^J^ilii-^i^P'i)'^' where ug is deterministic 



and stands for the sample size at the i-th stage. Suppose that sampling is continued until \fig\ > 
*7i^-i, _|_ some stage with index i. Define estimator p, = Jii, where I is the index of 



stage at which the sampling is terminated. Then, Pr{l < oo} = 1 and Pr{|/i — fi\ < e\fi\} > 1 
for any /x G (— oo, 0) U (0, oo) and a G (0, oo) provided that 2(t + 1)(" < 1 and inf^>o 



"{■+1 



> 1. 



See Appendix IK. 41 for a proof. 



8.3 Control of Relative and Absolute Errors 

In some situations, it may be appropriate to estimate /x with a mixed error criterion specified by 
£a > and £r > 0. In this respect, we have 

Theorem 58 Define 6g = 6 for 1 < ^ < t and 6g = 52'^~^ for i > t, where t is a positive 
integer. For £ = 1,2, ■ ■ ■ , let Jig = ^' and ag = ^^j^ YllLi — P-t)^ ; where ug is determin- 
istic and stands for the sample size at the £-th stage. Suppose that sampling is continued until 
max i^ai Tt^) — ^"'^^/wi^^ f'^^ some stage with index £. Define estimator p, = pi, where I is the 
index of stage at which the sampling is terminated. Then, Pr{|/i — fi] < Ea or \p — fi\ < £^1^1} > 1 — (5 
for any fi G (— oo, oo) and a G (0, oo) provided that 2(r + 1)( < 1 and inf£>o ^^^^ > 1. 

See Appendix IK. 5 1 for a proof. 



9 Estimation of Exponential Parameters with Relative Precision 

For completeness, we shall discuss the estimation of the parameter of an exponential distribution. 
The material presented in this section is well known (see, [18] and the references therein). 
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Let X be a random variable of probability density function fx{x) = ^exp(— |), it is a 
frequent problem to estimate 6 based on i.i.d. samples Xi,X2, • • • oi X. Let < e < 1. Define 
= — "^^^ godl is determine the minimum sample size n such that 

< e I ^1 > 1 - 5 (37) 

for any > 0. For simplicity of notations, define Y = . Note that Y has a chi-square 

distribution of 2n degrees of freedom and that 

< e I 6l| = Pr{2n(l -e) <Y < 27i(l + e)} = 5'p(n - 1, n(l - e)) - S'p(n - 1, n(l + e)) 

for any > 0. Therefore, the minimum sample size to ensure (j37p is the minimum integer n such 
that 5p(n — 1, n(l — e)) — Sp{n — 1, n(l + e)) > 1 — 6, which can be easily computed. 





10 Exact Bounded- Width Confidence Intervals 

A classical problem in sequential analysis is to construct a bounded-width confidence interval 
with a prescribed level of coverage probability. Such a problem can be solved in our framework 
of multistage estimation described in Section 12.11 Specifically, the problem of constructing a 
bounded-width confidence interval can be formulated as the problem of constructing a random 
interval with lower limit =Sf (0, n) and upper limit ^ {6, n) such that ^ {0, n) — ^{0, n) < 2e and 
that Pr{^(0,n) < 9 < ^ {6,n) \ 6} > 1 — 5 ior any 6 £ Q. For this purpose, our computational 
machinery such as bisection coverage tuning and AMCA can be extremely useful. 

10.1 Construction via Coverage Tuning 

As an application of Theorem [2l our general theory for constructing bounded-width confidence 
intervals based on multistage sampling is as follows. 

Corollary 5 Suppose a multistage sampling scheme satisfies the following requirements, 
(i) For £=!,■■■ ,s,di is a ULE of 9. 

(a) For i = 1, - ■ ■ ,s, {J^{6i, n^) < 9i < ^ (Og, n^)} is a sure event. 

(Hi) For £ = 1, • • • , s, decision variable assumes value 1 if'^{9i, n^) — ^{Og, n^) < 2e and 
assumes value otherwise. 

(iv) {r>, = l}C {Fg^ (§,,^(g,,n,)) <CSi, Gg^ (g,,if(gf,n,)) < C^^} for £ = 1, ■ ■ ■ , s. 

(v) {'^ {6 s^Yis) — ^(^siHs) < 2e} is a sure event. 

Define ^(0,n) = J^{Oi,rYi) and ^(0,n) = {Oi^rvi), where I is the index of stage when the 
sampling is terminated. Then, '^{0,n) — ^(0,n) < 2e and 

s s 

Pr{^(e,n) >e\0} < ^Pr{J^(0,,n,) > 0, De ^ I \ 9} < (^,^1, 

1=1 e=i 

s s 

Pr{'^(S,n) <e\9} < ^Pr{<^(0£,nf) <9, De = l\e} <(:^6e 

£=1 1=1 
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and Pr{^(0, u) < 6 < {d^n) \ 0} > I - 2^ ELi for anyOeQ. 

10.2 Bounded- Width Confidence Intervals for Binomial Parameters 

In this subsection, we provide concrete multistage sampling schemes for the construction of 
bounded- width confidence intervals for binomial parameters. 



10.2.1 Construction from Clopper-Pearson Intervals 

Making use of Corollary [5] and the Clopper-Pearson confidence interval jl6| , we have established 
the following sampling scheme. 



. For 



Corollary 6 Let < e < ^. Suppose the sample size at the s-th stage is no less than 
i = 1, • • • , s, let ^(j)g,n£) be the largest number such that < ^{p^^n^) < p^, 1 — SBiniPi — 
l,n£, ^{pi,ni)) < (5 and let '^{p£,ni) be the smallest number such that p^ < '^{p^jUi) < 

1, S-Bin£p^,n£,^ {pg,n£)) < (6, where p^ = — -. For £ = ,s, define Di such that 

Di = 1 if ^ {pi,ni) — ^{pi,ni) < 2e; and Di = otherwise. Suppose the stopping rule is that 
sampling is continued until Di = 1 for some £ E {!,••• ,s}. Define ^(p, n) = Jif{pi,ni) and 
^ {p, n) = f/ {pi , ni ) with p = Pi and n = ni, where I is the index of stage when the sampling is 
terminated. Then, ^ (p, n) — -Sf (p, n) < 2e, 



£=1 
s 

Prl'^O, n)<p\p}<Y, Pr{^(P£, ng) <p, De^l\p}< 3(6 



e=i 



and Pr{^(0, n) < p < ^(0, n) | p} > 1 - 2sC6 for any p G (0, 1). 

Based on the criteria proposed in Section 12. H the sample sizes ni < n2 < 
chosen as the ascending arrangement of all distinct elements of 



Cr-e In ^ 
2? 



£ = 1, ■ 



< n, can be 



(38) 



where r is the maximum integer such that 



2e^ 



, i.e., Cr-i > 



2s^ 



ln(l-2e) 

10.2.2 Construction from Fishman's Confidence Intervals 

Making use of Corollary [5] and Chernoff-Hoeffding inequalities [14:\ [27] , we have established the 
following sampling scheme. 



Corollary 7 Let < e < ^. Suppose the sample size at the s-th stage is no less than 



In - 



2e2 



. For 



£ = 1, - ■ ■ ,s, let =2'(p£, ni) be the largest number such that < .^{Pi, ni) < p^, {Pe, -^{Pe, rii)) < 
^^i^ and let {p^,ni) be the smallest number such that p^ < {pg,ni) < 1, (p^,'^ {p^,ni)) < 



-, where Pi 



For 



,s, define Di such that = 1 if 'W{pi,ne) 
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££{pi^ ni) < 2e; and Di = otherwise. Suppose the stopping rule is that sampling is contin- 
ued until Dg = 1 for some i G {1, • • • , s} . Define ^(p, n) = ^{pi,ni) and ^{p,n) = ^{pi,ni) 
with p = Pi and n = ni, where I is the index of stage when the sampling is terminated. Then, 
'^(p,n) -^(p,n) < 2e, 

S 

1=1 

s 

Vr{^{p,n) <p\p} <^Vr{^^{p^,n() <p, Di,^\\p}< 3(6 
e=i 

and Pr{^(0, n) < p < 'W(d,n) \ p} > 1 - 2sC6 for any p G (0, 1). 

Based on the criteria proposed in Section I2.H the sample sizes ui < n2 < ■ ■ ■ < Ug can be 
chosen as the set defined by p8]) . 



10.2.3 Construction from Explicit Confidence Intervals of Chen et al. 

The following sampling scheme is developed based on Corollary [5] and the explicit confidence 
intervals due to Chen et al 1121. 



Corollary 8 Let < e < ^. Suppose the sample size at the s-th stage is no less than r|(^+l)(^ — 

1) In ^] . For £ = !,■■■ ,s, define pg = ^^^^ — - and such that Di = 1 if l~ 21^^ Pi (1 - Pi) < 
r 1 ^ 

I ~ 2 in"c<5) ' ~ ^ otherwise. Suppose the stopping rule is that sampling is continued 

until Di = 1 for some i G {!,• • • , s}. Define 

1 - 2P£ - J^-2^Pii^~Pi) 



J^'iPg, ni) ^ max < 0, + - 



4 1 



81n(C<5) 



{Pi. n() = mm <( 1, + - 9„, 

8 ln(C<5) 



7 



for i = 1, • • • , s and p = Pi and n = ni, where I is the index of stage when the sampling is 
terminated. Then, (p, n) — .if (p, n) < 2e and 

s 

Pr{J^ip, n)>p\p}<Y, M^iPe, ^e) >p, D(^l\p}< 3(6, 

£=1 

Pr{'^(p, n)<p\p}<J2 M^iPe, ri,) < p, De = 1 \ p} < 3(6 

1=1 

for any p G (0, 1). 

Based on the criteria proposed in Section I2.H the sample sizes ni < n2 < ■ ■ ■ < Us can be 



chosen as the ascending arrangement of all distinct elements of | Cr-e ~ I) 1^ ^ 
where r is the maximum integer such that Cr-i — |) In ^ > (]^ ~ |) In i.e., Cr-i > 3+4£- 



62 



10.3 Bounded- Width Confidence Intervals for Finite Population Proportion 

In this subsection, we consider the construction of bounded-width confidence intervals for finite 
population proportion, p, based on multistage sampling. Within the general framework described 
in Sections 12.11 and 12.61 we have established the following method by virtue of Corollary [5] for 
bounded-width interval estimation. 

Corollary 9 For z G : < k < n}, define ^{z,n) = uim{z, L{z,n)} and '^{z,n) = 
max{z,U{z,n)} , where L{z,n) = mm{6 G 6 : 1 — SN{nz — l,n,9) > (6} and U{z,n) = max{0 e 
6 : SN(riz,n,9) > C,5}. Suppose the sample size at the s-th stage is no less than the smallest num- 
ber n such that (z, n) — ^(z, n) < 2e for all z d : ^ < k < n} . For £ = 1, • • • ,s, define 

Pe = — - and decision variable Di which assumes values 1 if {p^^Uf) — ^{p^^ng) < 2e and 
value otherwise. Suppose the stopping rule is that sampling is continued until Di = 1 for some 
i £ {!,•■■ ) s}. Define p = Pi and n = ui, where I is the index of stage when the sampling is 
terminated. Then, ^ (p, n) — ^(p, n) < 2e, 

Pr{^(p, n) > p I p} = Pr jif (p, n) - ^ > p | p| < ^ Pr n,) - ^ > p, = 1 | p| < <(5, 

Pr{'^{p, n)<p\p} = Pr |^(p,n) + 1 < p | p| < ^ Pr + < p, Di ^ 1 \ p'j < s^S 

and Pr{^(p, n) < p < '^(p, n)} > 1 - 2sC6 for all p G 6. 
Let 

'^max be the smallest number n such that ^ (z, n) — n) < 2e for all z £ : < /c < 
n}. Let njnin be the largest number n such that (z, n) — ^(z, n) > 2e for all z G : < A; < n}. 
Based on the criteria proposed in Section 12.11 the sample sizes ni < 77-2 < • • • < can be chosen 
as the ascending arrangement of all distinct elements of {[Cr- £ ^imaxl : ^ = 1, ■ ■ ■ , '''}) where r is the 
maximum integer such that C^-i > . 

In order to develop multistage sampling schemes with simple stopping boundaries, we have 
the following results. 

Corollary 10 For z £ : < k < n}, define J£{z,n) = inm{z, L{z,n)} and '^{z,n) = 
ma,x{z,U{z,n)} , where L{z,n) ~ m\n{9 <E 9 : C{z,9,n, N) > (6} and U{z,n) — ma,x{9 e Q : 
C{z,9,n, N) > (6}, where C{z,6,n, N) is defined by (3^. Suppose the sample size at the s-th stage 
is no less than the smallest number n such that {z, n)— ^(z, n) < 2e for all z €z : < k < n}. 

For £ = I,-- - ,s, define = — — - and decision variable which assumes values 1 if 
^{Pi,ni) — ^{pi,ni) < 2e and value otherwise. Suppose the stopping rule is that sampling is 
continued until Di = 1 for some £ S {1, • • • , s}. Define p = Pi and n = ui, where I is the index 
of stage when the sampling is terminated. Then, {p, n) — =Sf (p, n) < 2e, 

Pr{^(p, n) > p I p} = Pr |if (p, n) - 1 > p | p| < ^ Pr | m) - ^ > p, = 1 | p| < 

Pr{^(p,n) <p|p} = Pr|^(p,n) + ^ <p|p| <^Pr|^(p„n£) + ^ <p, = 1 | p| < <5 
and Pr{^(p, n) < p < '^(p, n)} > 1 - 2sC6 for all p G G. 
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Corollary 1101 can be shown by virtue of Corollary [5] and inequalities (j35p and (j36|) . 

11 Estimation Following Multistage Tests 

When a multistage hypothesis test is finished, it is usually desirable to construct a confidence 
interval for the unknown parameter 9. In general, multistage test plans can be cast in the 
framework of sampling schemes described in Section 12. li We have established various interval 
estimation methods in the context of multistage tests. 

11.1 Clopper-Pearson Type Confidence Intervals 

Define cumulative distribution functions Fg{z, 9) and Gg{z, 9) as (|2.5p . To construct a confidence 
interval of Clopper-Pearson type following a multistage test, we have the following results. 

Theorem 59 For £ = 1, - ■ ■ ,s, let 6^ = <^(Xi, • • • , ) be a ULE of 9. Let 6 = 6i and n = ni, 

where I is the index of stage when the sampling is terminated. Define confidence limits .^{0, n) 
and ^(0,n) as functions of {6,n) such that, for any observation {9,n) of {0,n), ^{9,n) is the 
largest number satisfying Gg{9,Ji'{9,n)) < | and that '^{9,n) is the smallest number satisfying 
Fg(9,^(9,n)) < |. Then, Pr{^(0,n) < 9 < "^(6,11) \ 9} > 1 - 6 for any 9 € G. 

See Appendix IL.ll for a proof. It should be noted that, by virtue of our computational 
machinery, exact computation of confidence intervals is possible for common distributions. 

11.1.1 Finite Population Proportion 

To construct a confidence interval for the proportion of a finite population after a multistage test 
in the general framework described in Sections 12.11 and 12.61 we have developed an approach which 
does not rely on using ULEs as follows. 

Theorem 60 Let = '^^ — - for £ = 1, • • • ,s. Let p = Pi and n = ni, where I is the index 
of stage when the sampling is terminated. Define confidence limits ^(p, n) and ^ (p, n) as 
functions of {p,n) such that, for any observation {p,n) of{p,n), J^{p,n) is the smallest number 
in Q satisfying Pr{p > p \ J!f{p,n)} > | and that ^{p,n) is the largest number in Q satisfying 
Pr{p < p\ '^{p, n)} > |. Then, Pr{^(p, n) < p < {p, n) \ p} > 1 — 6 for any p £ Q. 

See Appendix IL. 2 1 for a proof. 

11.2 Confidence Intervals from Coverage Tuning 

The method of interval estimation described in Section [11.11 suffers from two drawbacks: (i) It is 
conservative due to the discrete nature of the underlying variable, (ii) There is no closed-form 
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formula for the confidence interval. In light of this situation, we shall propose an alternative 
approach as follows. 

Actually, it is possible to define an expression for the confidence interval such that the lower 
confidence limit ^ and upper confidence limit ^ are functions of confidence parameter (5, coverage 
tuning parameter C, and 6 = Oi, where I is the index of stage when the sampling is terminated 
and £ = !,■■■ ,s are ULEs as defined in Theorem [59l Suppose ^{0, n) < < ^{6, n) and 

PT{e < ^(Oe, ne) I 9} < CSe, > n,) | 6} < C6i 

for £=!,••• , s. Then, 

s s 

Fi{e < if (0, n)\9}<Y, MO < ^{Oi, ne), De = i\e}<cYl S,, 

s s 

Fi{9 > ^(d,n) I 9} < J^Pr{0 > "^(6^,^), De = l\9}< C ^^5^. 

e=i e=i 
This implies that it is possible to apply a bisection search method to obtain a number C such that 
the coverage probability is no less than 1 — 6. For the purpose of searching we have established 
tight bounds for Pr{^(0,n) < 9 < "^{6,11) \ 9} for 9 e [a,b] C e as in Section [331 By virtue 
of such bounds, adaptive maximum checking algorithm described in Section 13.31 can be used to 
determine an appropriate value of C- 

11.2.1 Poisson Mean 

At the first glance, it seems that the approach described at the beginning of Section 111.21 cannot 
be adapted to Poisson variables because the parameter space is not bounded. To overcome such 
difficulty, our strategy is to design a confidence interval such that, for a large number A* > 0, 
the coverage probability is always guaranteed for A G (A*, 00) without tuning the confidence 
parameter and that the coverage probability for A G (0, A*] can be tuned to be no less than 1 — 6. 
Such method is described in more details as follows. 

Suppose the multistage testing plan can be put in the general framework described in Section 
12.11 Let a € (0, 1) and = — — -. For every realization, (A^, n^), of (A^, n^), let L = L{Xi, ni, a) 
be the largest number such that L{Xi,n£,a) < Xi and Pr{A^ > Xi \ L} < a. Let U = U{Xi,n£,a) 
be the smallest number such that U{X£,ni,a) > Xi and PrjA^ < Xe, \ U} < a. One possible 
construction of L and U can be found in [TS]. To eliminate the necessity of evaluating the 
coverage probability of confidence interval for an infinitely wide range of parameter A in the 
course of coverage tuning, the following result is crucial. 

i/[/(A,,n,,|) < A*, 
i/?7(A,,n,,|) > A* 
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Theorem 61 Define 

^(A,,n,) 




and 



U{Xe,ne,CS) i/C/(A,,n,,^) < A* 



2s ' 

C/(Af,n,,|) 2/C/(A,,n,,^) > A 



Let the lower and upper confidence limits be, respectively, defined as ^(A, n) = ^(Aj,nj) and 
'^(A,n) = '^{Xi,ni), where I is the index of stage when the sampling is terminated. Then, 

Pr{^(A,n) < A < '^(A,n) I A} > 1-5 (39) 

for any A G (0, oo) provided that (3^ holds for any A G (0, A*]. 

See Appendix IL.3I for a proof. 



11.2.2 Normal Variance 

A wide class of test plans for the variance of a normal distribution can be described as follows: 

Choose appropriate sample sizes ni < n2 < • • • < and numbers ag < bi, I = 1, ■ ■ ■ , s. Let 
a I = ^ ^ Yl^=i{-^i ~ ^niY for £ = 1, • • • ,s. Continue sampling until < ai or ai > bg. When 
the sampling is terminated, accept Ji^ if ag < af, reject Ji^ if ag > bg. 

To construct a confidence interval for a after the test, we can use a ULE of cj, which is given 
by = ai, where I is the index of stage when the test is completed. Accordingly, n = is the 
sample number when the test is completed. A confidence interval with lower limit ^{a,n) and 
upper limit {a, n) can be constructed as follows: 

If a assumes value a at the termination of test, the realization of the upper confidence limit 
is equal to a certain value a such that Pr{3r < | a} = |. Similarly, the realization of the lower 
confidence limit is equal to a certain value a such that Pr{or > 5 | o"} = |. 

To find the value of a such that Prj? < a | a} = |, it is equivalent to find a such that 

s 

Fr{a <a\a} = ^Fi{ae<a, aj <aj <bj, I < j < i \ a} . (40) 
Similarly, to find the value of a such that Pr{or > a | cr} = |, it is equivalent to find a such that 

s 

Pr{5 > 5 I ct} = ^ Pr > CT, aj <aj <bj, I < j < i \ a} . (41) 

e=i 

If we choose the sample sizes to be odd numbers ng = 2kg + 1, £ = 1, • • • , s, we can rewrite (I40p 
and ([4T]) respectively as 

1=1 \m=l ^ m=l ^ ^ I 

(42) 
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and 

p.(^>5l.)=tp4^:^■»^Y(^)^l(f^<Ea,.<|(^)^orl<i<^l4■ 

e=l \^rn=l ^ ^ m=l ^ ^ I 

(43) 

where Zi,Z2,--- are i.i.d. exponential random variables with common mean unity. As can 
be seen from ()42p and ()43p . the determination of confidence interval for a requires the exact 
computation of the probabilities in the right-hand sides of (|42p and (143p . For such computational 
purpose, we can use Theorem [311 



11.2.3 Exponential Parameters 

A wide class of test plans for the variance of a normal distribution can be described as follows: 

Choose appropriate sample sizes ni < n2 < • • • < and numbers < bi, i = 1, • • • , s. 
Define 6i = ^1 — ' ^ ~ Continue sampling until < ac or Oi > bg. When the 

sampling is terminated, accept J% if Oi < a^; reject J% if Oi > bg. 

To construct a confidence interval for 9 after the test, we can use a ULE of 6, which is given 
hy 6 = 6i, where I is the index of stage when the test is completed. Accordingly, n = is the 
sample number when the test is completed. A confidence interval with lower limit -Sf (0, n) and 
upper limit {6, n) can be constructed as follows: 

If 6 assumes value 9 when the test is completed, the realization of the upper confidence limit 
is equal to a certain value 9 such that Pr{0 < 9 \ 9} = ^. Similarly, the realization of the lower 
confidence limit is equal to a certain value 9 such that Pt{6 > 9 \ 9} = ^. 

To find the value of 9 such that Pr{0 < | 0} = |, it is equivalent to find 9 such that 

s 

Pr{d < ^ I 0} = ^ Pr < ^, aj <dj<bj, 1 < j < £ | . (44) 
Similarly, to find the value of 9 such that Pr{0 > | 0} = |, it is equivalent to find 9 such that 

s 

Pr{0 >9\9} = ^PTi^di>9, Uj <dj<bj, 1 < i < £ I . (45) 
e=i 

Let Zi,Z2,--- be i.i.d. exponential random variables with common mean unity. Then, we can 
rewrite (jli]) and (jl5|) respectively as 

Pr{0 < ^1 0} = ^Pr| < U J , n, (^) < < n, (^-A for 1 < j < £ \ e\ (46) 



£—1 K,m—1 \ / m— 1 

and 



PT{d>e\0} = Y.PT\Y,Z„.>ne[-), nJ^) < ^ Z„ < U for 1 < < ^ | . (47) 



^ 7n—l 



As can be seen from ()46p and (I47p . the determination of confidence interval for a requires the exact 
computation of the probabilities in the right-hand sides of (|46p and (I47p . For such computational 
purpose, we can make use of the results in Theorem! 
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12 Exact Confidence Sequences 



The construction of confidence sequence is a classical problem in statistics. In this section, we 
shall consider the problem in a general setting as follows. 

Let Xi,X2, - ■ ■ be a sequence of samples of random variable X parameterized hy 6 G O. 
Consider a multistage sampling procedure of s stages such that the number of available samples 
at the ^-th stage is a random number for £ = 1, • • • , s. Let 9i be a function of random tuple 
Xi, • • • , for £ = !,••• , s. The objective is to construct intervals with lower limits ^{6i^ n^) 
and upper limits ^ {9i, n^) such that 

Pr{^(0£,n^) < ^ < ^(0^,n£), £ = 1, ■ ■ ■ , s \ 0} > 1 - 6 

for any 9 & Q. 

12.1 Construction via Coverage Tuning 

Assume that 0^ is a ULE for £ = !,••• , s. For simplicity of notations, let 

Li = ^(ei,ni), Ui = ^ide,n^), i = l,---,s. 

As mentioned earlier, our objective is to construct a sequence of confidence intervals {Lg, Ui), 1 < 
^ < s such that Pr{L^ < 9 < Ue, 1 < £ < s | 6*} > 1 - 5 for any 6* G 6. Suppose 

FT{Le <9 <Ui\9}>l-C6, l<e<s 

for any 9 € @. By Bonferroni's inequality, we have Pr{L^ < 9 < Ui, l<^<s|^}>l — sC,6 
for any G 0. This implies that it is possible to find an appropriate value of coverage tuning 
parameter ( such that FrjL^ < 9 < Ui, 1 < £ < s | 6*} > 1 — 5 for any G 0. 

For this purpose, it suffices to bound the complementary probability 1 — Pr{L^ < 9 < Ui, 1 < 
i < s \ 9} and apply the adaptive maximum checking algorithm described in Section 13.31 to find 
an appropriate value of the coverage tuning parameter ^ such that 1 — Pr{L^ < 9 < Ue, 1<^< 
s \ 9} < 5 for any 9 G [a, b] C 0. In this respect, we have 

Theorem 62 Let Xi,X2, ■ ■ ■ be a sequence of identical samples of discrete random variable X 
which is parameterized by 9 £ Q. For i = 1, ■ ■ ■ , s, let = ip{Xi, ■ ■ ■ , X^i) be a ULE of 9. Let 
Li = ^(6i,n£) and Ug = '^{9i,ni) be bivariate functions of Og and such that {Li < 6 < 
Ui}, i = 1, ■ ■ ■ , s are sure events. Let [a,b] be a subset ofQ. Let denote the intersection of 
[a,b] and the union of the supports of Li, i = ,s. Let U^ denote the intersection of [a,b] 

and the union of the supports of Ug, i = ,s. Define 

s 

Pl{9) = 5^Pr{Lfc >9, L,<9<Ui,l<l<k\9], 

k=l 

s 

Pu{0) = J^Pr{[/fc <9, L,<9 <Ui, l<l<k\9}. 

k=l 
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The following statements hold true: 

(I) : 1 - Fr{Le <e<Ui, 1 < i < s \ 9} = + PuiO). 

(II) : Pl{9) is non- decreasing with respect top £ Q in any interval with endpoints being consecu- 
tive distinct elements o/ U {a, 6}. The maximum of Pl{0) over [a,b] is achieved at Ij^L) {a,b} . 
Similarly, Pu{9) is non-increasing with respect to p £ Q in any interval with endpoints being 
consecutive distinct elements of If/ L){a,b}. The maximum of Pu{9) over [a,b] is achieved at 

U {a,b}. 

(III) : Suppose that {Lg > a} C {0£ > 6} and {Ui < 6} C {0£ < a} for £ = 1, - ■ ■ ,s. Then, 

s 

Pl{0) < J2^^i^k> a, Le <b, Ue > a, I < e < k\b}, 

k=l 

s 

Pu{0) < ^ Pr{t/fe <b, Li,<b, Ui> a, l<£<k\a], 

k=l 

s 

Pl{0) > ^VT{Lk>b, Li<a,Ui>b, l<£<k\a], 

k=l 

s 

Pu{0) > ^'P^{Uk<a, Le<a, Ui>b, l<i<k\b} 

k=l 

for any 9 £ [a,b] C Q. 

Theorem [62] can be established by a similar argument as that of Theorem [3l It should be 
noted that no need to compute s terms in the summation independently. Recursive computation 
can be used. 



12.2 Finite Population Proportion 

To construct a confidence sequence for the proportion, p, of a finite population described in Section 
12. H we have the following results. 

Theorem 63 Let = ^(pg,ni) and Ue = ^{p£,ni) be bivariate functions ofp^ = ^=^^^ — - and 
n£ such that < < Ug and that both NLg and NUg are integer-valued random variables for 
£ = ,s. Let a < b be two elements of Q = : m = 0, ,A^}. Let I^ denote the 

intersection of (a, 6) and the union of the supports o/ — £ = 1, ■ ■ ■ , s. Let Ii^ denote the 
intersection of (a, 6) and the union of the supports ofUi + j^, £ = !,••• , s. Define 

s 

Pl{p) = J^Pr{Lfc >p, Lg<p<Ui, l<£<k\p], 

k=l 

s 

Pu{p) = Yl P^i^'^ <P, Le<p<Ui, l<£<k\p}. 

k=l 

The following statements hold true. 
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(I) : 1 - Pi{Le <p<Ue, l<i<s\p} = Pl{p) + Pu{p)- 

(II) : Pl{p) is non- decreasing with respect top £ @ in any interval with endpoints being consecu- 
tive distinct elements of I^L) {a,b} . The maximum of Pl{p) over [a,b] is achieved at I^L) {a,b} . 
Similarly, Pu{p) is non-increasing with respect to p £ Q in any interval with endpoints being 
consecutive distinct elements of U{a,b}. The maximum of Pu{p) over [a, 6] is achieved at 
% U{a,&}. 

(III) : Suppose that {Li > a} C {p^ > b} and {Ue < 6} C {p^ < a} for £ = !,••• ,s. Then, 

s 

Pl{p) < ^Pr{Lfc > a, < > a, 1 < ^ < A; I 6}, 

k=l 

s 

Pu{p) < ^Pr{[/fc<6, L£<6, [/^>a, l<£<A;|a}, 

k=l 

s 

Pl{p) > ^Pr{Lfc>6, L£<a, C/£>6, l<£<fc|a}, 

k=l 

s 

Pu{p) > Yj ^^{^k <a, Li<a, Ui>b, l<l<k\b} 

k=l 

for any p G [a, b] n 0. 

Theorem [63] can be established by a similar argument as that of Theorem [2j It should be 
noted that our computational machinery such as bisection coverage tuning, AMCA and recursive 
algorithm can be used. 



12.3 Poisson Mean 

At the first glance, it seems that the adaptive maximum checking algorithm described in Section 
13.31 cannot be adapted to Poisson variables because the parameter space is not bounded. To 
overcome such difficulty, our strategy is to design a confidence sequence such that, for a large 
number A* > 0, the coverage probability is always guaranteed for A G (A*,oo) without tuning the 
confidence parameter and that the coverage probability for A G (0, A*] can be tuned to be no less 
than 1 — 6. Such method is described in more details as follows. 

Let a G (0, 1) and = — — -. For every realization, (X£,ni), of (A^,n£), let L = L{\£,n£,a) 
be the largest number such that L{Xi,ni,a) < Xe and Pr{A^ > A^ | L} < a. Let U = U{Xe,ne,a) 
be the smallest number such that U{X£,ni,a) > Xi and PrjA^ < Xi \ U} < a. One possible 
construction of L and U can be found in [19]. To eliminate the necessity of evaluating the 
coverage probability of confidence interval for an infinitely wide range of parameter A in the 
course of coverage tuning, the following result is critical. 

i/[/(A,,n,,|) < A*, 
i/?7(A,,n,,|) > A* 



Theorem 64 Define 

L(\e,ni,C5) 



^(A,,n,) 
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and 

V(A,,n,,C5) i/C/(A,,n,,A) < A* 



Then, 



C/(Af,n,,|) 2/C/(A,,n,,^) > A 



Pr{Jf (A,, n,) < A < '^(A,, n,), i = 1, ■ ■ ■ , s \ X} > 1 - 6 (48) 
/or any A £ (0, oo) provided that ^8\ ) holds for any A € (0, A*]. 

See Appendix [M] for a proof. 

12.4 Normal Mean 

For normal variable, we have 

Pr{X„^ - Zi^s cr/ V^i < /i < + 2(6 cr/ V^i, l<^<s}>l - sC5. 
Hence, if we choose C to be small enough, we have 

Pr{X„^ - Zi^s cr/ < fj.< + Zc_s o j ^/nl, l<i<s} = l-5. 

To compute the coverage probability of the repeated confidence intervals, there is no loss of 
generality to assume that Xi,X2, ■ ■ ■ are i.i.d. Gaussian variables with zero mean and variance 
unity (i.e., /u = 0, (7 = 1). Hence, it suffices to compute Pr{|X„J < Zi^^j^fnl^ 1 < ^ < s}- We 
shall evaluate the complementary probability 

1 - Pr{|X„J < Z(;5/^n7, 1 < ^ < s} = Pr{|X„J > Z^a/^/n^for some £ among 1, • • • ,s} 

= ^Pr{|X„J > Z^a/V^and |X„J < Zc5/^/^, \ <l<r} 

r=\ 
s 

= 2 Pr{X„^ > Zc^^l^r and |X„, | < Z^s/V^l. l<i<r}. 

r=l 

Hence, the bounding method based on consecutive decision variables described in Section (22] can 
be used. Specifically, 

1 - Pr{|X„J < Z^s/V^i, 1 < ^ < s} 

s 

< 2^Pr{X„,, > Zc^^l^/rTr and |X„,J < Zc_s/Vnli max(l,r - k) < I < r} 

r=l 

for 1 < A; < s. Such method can be used for the problem of testing the equality of the mean 
response of two treatments (see, [33], [39] and the references therein). It can also be applied to 
the repeated significance tests established by Armitage, McPherson, and Rowe [2]. 
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12.5 Normal Variance 



In this section, we shall discuss the construction of confidence sequence for the variance of a 
normal distribution. Let X\,X2, • • • be i.i.d. samples of a normal random variable X of mean /i 
and variance cr^. Our method of constructing a confidence sequence is follows. 

Choose the sample sizes to be odd numbers = 2ki + 1, £ = !,••• , s. Define 
and = X]i=i(^« ~ for £ = 1, • • • , s. Note that 



2 

-^n^— 1,1— ^rn~l,CS 



{ <o^ < ^ , 1 < £ < s ^ > 1 - 2sC5 



and 



where .Z^i, Z2, • • • are i.i.d. exponential random variables with common mean unity. There- 
fore, the coverage probability Pr < 2 "^"^ < cr^ < -^^^ — , 1 < ^ < s > can be exactly computed 

by virtue of Theorem [SH Consequently, we can obtain, via a bisection search method, an appro- 
priate value of C, such that 

[Xnf-l,l-C5 ^nf-l,C5 J 

12.6 Exponential Parameters 

In this section, we shall consider the construction of confidence sequences for the parameter Q of 
a random variable X of density function /(x) = ^ exp (— f ) . Let Xi, X2, • • • be i.i.d. samples of 
a normal random variable X. Let ni < n2 < • • • < ns be a sequence of sample sizes. Since 
has a chi-square distribution of In degrees of freedom, we have 



< ^ < s ^ > 1 - 2sCb, 



or equivalently. 



2 ^ V ^ 2 

Note that 
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where Zi, Z2, • • • are i.i.d. exponential random variables with common mean unity. There- 

2 — - < 9 < i='^ — 1 < ^ < s > can be exactly computed by 
virtue of Theoreml54[ Consequently, we can obtain, via a bisection search method, an appropriate 
value of C such that 



6. 



13 Multistage Linear Regression 

Regression analysis is a statistical technique for investigating and modeling the relationship be- 
tween variables. Applications of regression are numerous and occur in almost every field, including 
engineering, physical sciences, social sciences, economics, management, life and biological sciences, 
to name but a few. Consider a linear model 

y = (3lXi + (32X2 -\ l-PmXm + W with Xi = 1, 

where /3i , • • • , are deterministic parameters and w is a Gaussian random variable of zero 
mean and variance <t^. A major task of linear regression is to estimate parameters a and /3j 
based on observations of y for various values of Xj. In order to strictly control estimation error 
and uncertainty of inference with as few observations as possible, we shall develop multistage 
procedures. To this end, we shall first define some variables. Let (3 = [/3i, • • • ,/3m]^, where the 
notation "j" stands for the transpose operation. Let 101,102, • " be a sequence of i.i.d. samples 
of w. Define 

Hi = /3iXii + l32Xi2 H h PmXim + Wi with Xii = 1 

for i = 1, 2, • • • . Let ni, i = 1,2, ■ ■ ■ be a sequence of positive integers which is ascending with 
respect to i. Define 



yi 
y2 



Xii 

X21 



X12 
X22 



X\m 
X2m 



Xrifl Xyi^2 ' ' ' Xnim_ 



for 



1,2,--- 



Assume that XjXe is of rank m for all i. Define 
Be = {X]Xe)-^XjYe, = 



rii — m 



for £ = 1,2, • • • . For i = 1, • • • ,m, let denote the i-th entry of Bi and let ^(X^X^) ^] .. 
denote the (i,i)-th entry of (XjXe)'^ . 
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13.1 Control of Absolute Error 



For the purpose of estimating the variance a and the parameters /3j with an absolute error criterion, 
we have 

Theorem 65 Let e > and e.i > for i = 1, • • • , m. Let t be a positive integer. Suppose the 
process of observing y with respect to Xi and w is continued until tm-m, cSe S-£ ^[{XjXe)^^].^ < Si 
for i = 1, • • • , m, and 

— m ^ ^ n£ — m ^ 
A — <^i - £ <<^i< J— 2 + e 

at some stage with index i, where = b for I < i < t and 5i = 52'^"^ for £ > t. Define 
a = ai and (3 = Bi, where I is the index of stage at which the observation of y is stopped. For 
i = 1, • • • , m, let (3^ be the i-th entry of p. Then, Pr{i < oo} = 1 and Prjl? — a\ < e, — < 
£i for i = 1, - ■ ■ , m} > 1 — 6 provided that 2{m + 1)(t + 1)C < 1 and that inf^^o > 1- 

See Appendix lN.il for a proof. 
13.2 Control of Relative Error 

For the purpose of estimating the variance a and the parameters /Sj with a relative error criterion, 
we have 

Theorem 66 Let < e < 1 and < < 1 for i = 1, • • • , m. Let t be a positive integer. Suppose 
the process of observing y with respect to xi and w is continued until tn^-m. C5f ^/[(XTXf)"!]^^ < 
■^^\Bi^i\ for z = 1, • • • , m, and "'^'^■^^y^^^ <ni~m< at some stage with index i, where 

be = S for I < £ < T and 6i = 52'^"^ for t > t. Define a = Bi and /3 = /3j, where I is the index 
of stage at which the observation of y is stopped. For i = 1, - ■ ■ ,m, let (3^ be the i-th entry of (3. 
Then, Pr{Z < oo} = 1 and Pr{|CT — o"| < ea, \j3^ — /3j| < £i\l3i\ fori = 1, • • • ,m} >1 — 5 provided 
that 2{m + 1)(t + 1)C < 1 and that inf^>o ^ > 1. 

See Appendix IN. 2 1 for a proof. 

14 Multistage Estimation of Quantile 

The estimation of a quantile of a random variable is a fundamental problem of practical impor- 
tance. Specially, in control engineering, the performance of an uncertain dynamic system can be 
modeled as a random variable. Hence, it is desirable to estimate the minimum level of perfor- 
mance such that the probability of achieving it is greater than a certain percentage. In general, 
the problem of estimating a quantile can be formulated as follows. 

Let X be a random variable with cumulative distribution function Fx{.). Define quantile 
= inf{x : Fx{x) > p} for p £ (0, 1). The objective is to estimate S^p with prescribed precision 
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and confidence level based on i.i.d. samples Xi, X2, • • • of X. To make it possible for the rigorous 
control of estimation error and uncertainty of inference, we shall propose multistage procedures. 
For this purpose, we need to define some variables. For an integer n, let Xi-n denote the i-ih order 
statistics of i.i.d samples Xi, - • • , Xn of X such that —00 = Xo-n < Xi-n < X2:n < • • • < Xnm < 
Xn+i:n = CO. Let the sample sizes be a sequence of positive integers n^, i = 1,2, ■ ■ ■ such that 
Til < n2 < < ■ ■ ■ ■ At the ^-th stage, the decision of termination or continuation of sampling is 
made based on samples Xi , X^ ■ 

14.1 Control of Absolute Error 

For estimating with a margin of absolute error e > 0, our sampling procedure can be described 
as follows. 

Theorem 67 For £ = 1, 2, • • • , define Se = 6 for 1 < £ < t and 6i = d2'^~^ for £ > t, where t 
is a positive integer. Let ii < he the largest integer such that YI^^Zq {^^^p^{l — p)^i~^ < (5^. 
Let ji > be the smallest integer such that Xlfcij^ Cfc)p'^(^ ~ p^^i-^ < (^,5^. Define such 
that Jp^^ = Xpnt.ni, ifpni is an integer and ^^^^ = {\pni \ - pn£)XLp„^j.„^ + {pni> - [pn^J )^rpnf]:n^ 
otherwise. Suppose that sampling is continued until Xj^-m — e < < -^ir-ne + ^ f'^^ some stage 
with index £. Define estimator = j where I is the index of stage at which the sampling is 
terminated. Then, Pr{Z < 00} = 1 and Pr{|^p — Cp\ ^ ^} ^ ^ ~ ^ provided that 2(t + 1)C < 1 and 
that inf,>o^ > 1. 

See Appendix 10.11 for a proof. 

14.2 Control of Relative Error 

For estimating 7^ with a margin of relative error e G (0, 1), our sampling procedure can be 
described as follows. 

Theorem 68 For £ = 1, 2, • • • , define 5i = 5 for I < i < t and 6£ = 52'^~^ for £ > t, where r is 
a positive integer. Let ii < be the largest integer such that Yl^kZo {"k)P^i^ — p^<^t-k < 
jl > &e the smallest integer such that X^^lj^ {^"'^)p^{l—p)^^~^ < C^i- Define such that = 
Xpnf.ni ifpne is an integer andip,, = {\pne] - pnt>)X\p^^y^^^ + {pm, - [p^-d )^rp"fl:"^ otherwise. 
Suppose that sampling is continued until [1 — sgn(^p £)e]Xj^:„^ < ^ < [1 + sgn(^p^^)e]Xi^:n^ for 
some stage with index £. Define estimator ^p = ^p i where I is the index of stage at which the 
sampling is terminated. Then, Pr{Z < cxd} = 1 and Pr{|^p — < e\^p\} > I — 6 provided that 
2(t + 1)C < 1 and that inf^>o ^ > 1- 

See Appendix 10.21 for a proof. 



75 



14.3 Control of Absolute and Relative Errors 



For estimating with margin of absolute error > and margin of relative error G (0, 1), 
our sampling procedure can be described as follows. 

Theorem 69 For i = 1, 2, • • • , define Se = 6 for 1 < £ < t and 6^ = 52"^"^ for £ > t, where r 
is a positive integer. Let ii < n£ be the largest integer such that Yl^k=o Ck)P^i^ — p"j^t-^ < ^J^. 
Let ji > be the smallest integer such that Xlfclj^ Cfc)p'^(^ ~ p^"-t-k < (^^^_ Define such 
that $p^i = Xpnt.ni, ifpne is an integer and ^p^^ = {\pni] - pn£)XLp„^j.„^ + {pn^ - [p^d )^rpnfl:nf 
otherwise. Suppose that sampling is continued until Xj^:ni — max(ea) ^S^i^p,e)^rXje:ni) < ^p/ ^ 
Xi^:ni + max(ea, sgn{^p^^)erXi^:ng) for some stage with index £. Define estimator ^p = ^p ^ 
where I is the index of stage at which the sampling is terminated. Then, Pr{Z < oo} = 1 and 
P'^{|^p~'?pl ^ or\^p — ^p\ < er\Cp\} > I — 6 provided that 2{t + \)C, < 1 and that infiyQ^^^^ > 1. 

See Appendix 10.31 for a proof. 

15 Conclusion 

In this paper, we have proposed a new framework of multistage parametric estimation. Specific 
sampling schemes have been developed for basic distributions. It is demonstrated that our new 
methods are unprecedentedly efficient in terms of sampling cost, while rigorously guaranteeing 
prescribed level of confidence. 



A Preliminary Results 
A.l Proof of Identity (H]) 



We claim that 



{\d-9\ <e,.|^|} 



c 



e 



1 + Sgn{6)£r 



<& < 



e 



1 - Sg-n.{e)£r 



Let uj e{\e-e\< £r.\e\] and 9 = e{uj). Then, \9 - 9\ < er\9\. To show 



< y < 



l+sgn(e)e, l-sgn(e)e,- ^ ^ 

In the case of > 0, we have 9 > (9 — £r\9\) > as a result of \9 



(49) 

it suffices to show 
< £r\9\. Moreover, 



<0 < 



l+sgn(6»)er 1+^-- " " " l-£r l-sgn(e)E 

result of |0 — ^1 < £r\9\. Moreover, 
established (H9]). 



In the case of ^ < 0, we have 9 < {9 + £r\9\) < as a 



l+sgn(e)e 



<0 < 



l+e,. 



l-sgn(e)£. 



Therefore, we have 



In view of (jl9]),it is obvious that{|0-0| < oi\e-9\ < £r\9\} C{^(0,n) <6'<^(0,n)}. 
To complete the proof of identity ([T|), it remains to show — ^| < or \6 — 9\ < £^1^1} 5 
{^{d,n) < 9 < 'W{d,n)}. For this purpose, let w G {^(0,n) < 9 < ^(0,n)} and 9 = d{u}). 
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Then, 



min <.6 - Sa, — > < < max <6 + £a, — > (50) 

1 l + sgn(^)ej 1 l-sgn(^)ej 



Suppose, to get a contradiction, that |^ — ^| > £a and |^ — ^| > Erl^l- There are 8 cases: 

(i) 9 > 0, i > 9 + £a, > 9 + £r\9\. In this case, we have ^ > 0, 9 < 9 - £a and 

l+sgn{0)er 



< jqr^ = 7-; — ^7^\ — 5 which contradicts the first inequahty of ()50p . 



(ii) 9 >0, 9 <9 — £a, 9 > 9 + £r\9\. In this case, we have 9 + er\9\ < 9 < 9 — £a, which imphes 
that Ea = and 9 > 0. Therefore, the first inequahty of ([50]) can be written as < 9, which 
contradicts to 6* > + £r\9\ = (1 + er)9. 

(iii) ^ > 0, 9 > 9 + £a, 9 < 9 — £r\9\. In this case, we have 9-\-£a < 9 < 9 — £r\9\, which implies 
that £a = and 9 > 0. Therefore, the second inequahty of (|50p can be written as > ^) which 
contradicts to 9 < 9 — £r\9\ = (1 — £r)9. 

(iv) 9 >0, 9 < 9—£a, 9 < 9—er\9\. In this case, we have > 9+ea and > jS^- Hence, by the 

second inequahty of (j50|) . we have < 9 < ^-^^ — , which imphes 9[1 — sgn{9)£r] < 9{1 — e,.), 

i.e., £r\9\ > £r9. It follows that ^ < and thus 9 < 0, which contradicts to > 0. 

(v) 9 < 0, 9 > 9 + £a, 9 > 9 + £r\9\. In this case, we have 9 < 9 — £a and 9 < Hence, by 
the first inequality of (1501) . we have -j-^ > 9 > — , which implies 9\l+ss,n(9)£r] > 9(1 — £r), 

Er l+sgn(y)er 

i.e., £r\9\ > —£r9. It follows that ^ > and thus ^ > 0, which contradicts to 6* < 0. 

(vi) 9 < 0, 9 < 9 — £a, 9 > 9 + £r\9\. In this case, we have 9 — £a > 9 > 9 + £r\9\, which implies 
that £a = and ^ < 0. Therefore, the first inequality of (|50p can be written as jS^ < ^) which 
contradicts to ^> 6^ + £r\9\ = (1 - £r)9. 

(vii) 9 <0, 9> 9 + £a, 9 < 9 — £r\9\. In this case, we have 9 — £r\9\ >9> 9 + £a, which implies 
that £a = and 9 < 0. Therefore, the second inequality of ([5U|) can be written as > 9, which 
contradicts to ^< 6^ - £r\9\ = (1 + £r)9. 

(viii) 9 < 0, 9 < 9 - £a, 9 < 9 - £r\9\. In this case, we have 9 < 0, 9 > 9 + £a and 
9 > TT— = ^-^ — , which contradicts the second inequality of (1501 ). 

From the above 8 cases, we see that the assumption that |^ — ^| > £a and \9 — 9\ > £r\9\ always 
leads to a contradiction. Therefore, it must be true that either |^ — ^| < £a or |0 — ^| < £r\9\- This 
proves {\0 — 9\ < £a or \0 — 9\ < £r\9\} 5 {Jf(0,n) < 9 < 'W{6,n)} and consequently completes 
the proof of identity ([1]) . 
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A. 2 Probability Transform Inequalities 

The well-known probability transform theorem asserts that Pr{F^(Z) < a} = Pr{G^(Z) < a} = 
a for any continuous random variable Z and positive number a G [0, 1]. In the general case that Z 
is not necessarily continuous, the probability transform equalities may not be true. Fortunately, 
their generalizations, referred to as "probability transform inequalities", can be established as 
follows. 

Lemma 2 Pr{Fz{Z) < a} < a and Pr{Gz{Z) < a} < a for any random variable Z and positive 
number a. 

Proof. Let Iz denote the support of Z. If {z £ Iz ■ Fz{z) < a} is empty, then, {Fz{Z) < a} is 
an impossible event and thus I'r{Fz{Z) < a} = 0. Otherwise, we can define z* = max{z G Iz '■ 
Fz{z) < a}. It follows from the definition of z* that Fz{z*) < a. Since Fz{z) is non-decreasing 
with respect to z, we have {Fz{Z) < a} = {Z < z*}. Therefore, Fr{Fz{Z) < a} = Pr{Z < 
z*} = Fz{z*) < a for any a > 0. By a similar method, we can show Pr{Gz{Z) < a} < a for any 
a > 0. 

□ 

A.3 Property of ULE 

Lemma 3 Let S be an event determined by random tuple {Xi, ■ ■ ■ Let (p{Xi, ■ ■ ■ be 

a ULE of 9. Then, 

(i) Vr{£ I ^} is non-increasing with respect to 9 £ Q no less than z provided that S C 
{(^(Xi,-- - < z\. 

(a) Pr{(f I 6} is non- decreasing with respect to 9 £ Q no greater than z provided that S C 
{(^(Xi,-- - ,X^) > z}. 

Proof. We first consider the case that Xi,X2, ■ ■ ■ are discrete random variables. Let Iyy^ denote 
the support of m, i.e., /m = {m{uj) : uj G VL}. Define = {{^li'^) ■, ' ' ' i^mi'^)) ■ ^ £ 
S", m(u;) = m} for m G /m- Then, 

Ft{^\9}=Y^ Fr{Xi = Xi, i = l,--- ,m\9}. (51) 

To show statement (i), using the assumption that C {ip(Xi,--- ,Xm) < z}, we have 
ip{xi,--- ,Xm) < z for (xi,--- ,Xm) G J%^m with m G Im- Since ip{Xi,--- ,Xm) is a ULE of 
9, we have that Pr{Xj = Xi, i = 1, ■ ■ ■ ,m \ 9} is non-increasing with respect to ^ G G no less 
than z. It follows immediately from (|5ip that statement (i) is true. 

To show statement (ii), using the assumption that S' C {(p(^Xi,--- > z}, we have 

(p{xi, • • • , Xm) > z for (xi, • • • , Xm) G with m G /m- Since (p{Xi, • • • , X^) is a ULE of 9, we 
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have that Pr{Xj = Xi, i = 1, ■ ■ ■ ,m \ 9} is non-decreasing with respect to G © no greater than 
z. It follows immediately from (j5ip that statement (ii) is true. 

For the case that Xi,X2, ■ ■ ■ are continuous random variables, we can also show the lemma 
by modifying the argument for the discrete case. Specially, the summation of likelihood function 
Pr{Xj = Xi, i = 1, ■ ■ ■ ,m \ 9} over the set of tuple (xi, • • • , Xm) is replaced by the integration of 
the joint probability density function fxi,-,Xm{^i^ ' ' ' ^Xm,0) over the set of (xi, • • • ,Xm)- This 
concludes the proof of Lemma [3l 

□ 

B Proof of Theorem [2] 

Making use of assumptions (ii)-(iii), the definition of the sampling scheme and the monotonicity 
of Fg^iz, 9) as asserted by Lemma [3l we have 

s 

Pr{e > ^(0,n) I 0} = ^Vi{9>'^(ei,ni),l = l\e] 

s 

< Y,PT[9>^(ee,ne)>de, ^(0^, n,)) < C<5H 

s s 

e=i i=i 
for any 9 £ Q, where the last inequality follows from Lemma [2j 

Similarly, we can show that Pr{6l < ^(6,n) \ 9} < CYle=i^e- Hence, Pr{^(^,n) < 9 < 
'^(0, n) I 6*} > 1 - Pr{6' < ^{0, n) \ 9} - Fv{9 > ^(0, n) | 6*} > 1 - 2C Ef=i ^i- This concludes 
the proof of Theorem [2l 

C Proof of Theorem [3] 

Let 9' < 9" be two consecutive distinct elements of U {a,b}. Then, {9 < ^{6,n) < 9"} C 
{9' < if (g, n) < 9"} = and it follows that {^{0, n) > 9} = {^{0, n) > 9"} U {9 < ^{6, n) < 
9"} = {^(e,n) > 9"} for any 9 G {9', 9"]. Recalling that {d > if(0,n)} is a sure event, we 
have {^(0,n) > 9"} = {6 > 9", ^{6,n) > 9"}. Invoking the second statement of Lemma 
El we have that Pr{6' < if (0, n) and (f occurs | 9} = Pr{if (S, n) > 9" and £' occurs | 9} = 
Pr{0 > 9", if (0, n) > 9" and occurs | 9} is non-decreasing with respect to € {9', 9"]. 
This implies that the maximum of Pr{9 < if (0, n) and S' occurs | 9} with respect to G 
{9', 9"] is equal to Pr{0 > 9", if(0,n) > 9" and occurs | 9"}. Since the argument holds for 
arbitrary consecutive distinct elements of I^U{a, b}, we have established statement (I) regarding 
Pr{^ < if (0, n) and ^ occurs | 9} for 9 € [a,b]. To prove the statement regarding Pr{^ < 
i^(0,n) and occurs | 9}, note that {9 < if(0,n) < 9"} C {9' < if(0,n) < 9"} = 0, which 



79 



implies that {^(d,n) > 9} = {^(e,n) > 9"} U {9 < < 9"} = {^{d,n) > 9"} for any 

9 E [9', 9"). Hence, Fr{9 < ^(e,n) and ^ occurs | 9} = Pr{^(0,n) > 9" and occurs | 9} = 
Fi{d > 9", ^{d,n) > 9" and ^ occurs | 9} is non-decreasing with respect to 6* G [9', 9"). This 
impUes that the supremum of Pr{9 < ^{9,n) and occurs | 9} with respect to G [9', 9") is 
equal to Pr{0 > 9", ^(9,n) > 9" and occurs | 9"}. Since the argument holds for arbitrary 
consecutive distinct elements of U {a,b}, we have established statement (I) regarding Pr{9 < 
^{0, n) and occurs | 9} for 9 G [a, b]. 

To prove statement (II) regarding Pr{0 > "^(0,11) and S' occurs | 9}, let 9' < 9" be two 
consecutive distinct elements oiIf/U{a,b}. Then, {9' < '^{d,n) < 9} C {9' < '^(0,n) < 9"} = 
and it follows that {^(0,n) < 9} = {^(^,n) < 9'} U {9' < ^(0,n) < 9} = {^(^,n) < 9'} 
for any 9 £ [9', 9"). Recalling that {6 < ^(0,n)} is a sure event, we have {'^(0,n) < 9'} = 
{9 < 9', "^{9,11) < 9'}. Consequently, Pr{'^(0,n) < 6* and occurs | 9} = Pr{'^(0,n) < 
9' and occurs | 9} = Pr{9 < 9', '^(9,n) < 9' and ^ occurs | 9} is non-increasing with 
respect to G [9', 9") as a result of the first statement of Lemma [3l This implies that the 
maximum of Pr{^(S, n) < 9 and occurs | 9} for 9 G [9', 9") is equal to Pr{S < 61', '^(9, n) < 
and £ occurs | 0'}. Since the argument holds for arbitrary consecutive distinct elements of 
/.^ U {a,fe}, we have established statement (II) regarding Pr{0 > '^(0,n) and S occurs | 9} for 
9 G [a, 6]. To prove the statement regarding Pr{^ > '^(0,n) and S occurs | note that {9' < 
'^{9,n) < 6*} C {9' < '^(?,n) < 9"} = 0, which implies that {^(0,n) < 9} = {'^(0,n) < 6*'} U 
{9' < ^(0,n) < 6'} = {^(0,n) < 9'} for any 6* G {9', 9"]. Hence, Pr{'^(0,n) < 6* and occurs | 
9} = Pr{'^(0,n) < 9' and occurs | 9} = Pr{0 < 9', '^(9,n) < 9' and S" occurs | (9} is 
non-increasing with respect to ^ G {9', 9"]. This implies that the supremum of Pr{^(0,n) < 
9 and ^ occurs | 0} for 9 G (e', is equal to Pr{0 < 0', ^(0, n) < 0' and occurs | ^'j. Since 
the argument holds for arbitrary consecutive distinct elements of U {a, b}, we have established 
statement (II) regarding Pr{^ > ^ (9, n) and S' occurs | 9} for 9 G [a, b]. 

To show statement (HI), note that Pr{9 < ^{9,n) and S' occurs | 9} is no greater than 
Pr{a < =Sf(0,n) and <^ occurs | 9} for any 9 G [a,b]. By the assumption that {a < ^{9,n)} C 
{9 > b}, we have Pr{a < ^{9, n) and <^ occurs | 9} = Pr{9 > b, a < ^{9, n) and S' occurs | 9} 
for any 9 G [a,b]. As a result of the second statement of Lemma O we have that Pr{0 > 
b, a < ^{9,n) and occurs | 9} is non-decreasing with respect to ^ G [a,b]. It follows that 
Fv{9 >b, a < ^{9, n) and (f occurs | 9} < Pi {9 > b, a < ^{9, n) and S' occurs | b} for any 
^ G [a, b], which implies that Pr{9 < .^{9, n) and occurs | 9} < Pr{a < -S?(0, n) and occurs | 
b} for any 9 G [a,b]. On the other hand, Fr{9 < ^{9,n) and S' occurs | 9} > Pr{6 < 
^{9,n) and S' occurs | 9} for any 9 G Recalling that {9 > ^{9,n)} is a sure event, 

we have Pr{& < ^(0,n) and S' occurs | 9} = Pr{6 < ^{9,n) < 9 and occurs | 9} 
for any 9 G [ajf*]- Hence, applying the second statement of Lemma [3l we have that Pr{6 < 
^(0,n) < 9 and S' occurs | 9} > Pr{6 < ^(0,n) < and (f occurs | a} = Pr{& < 
^{9,n) and occurs | a} for any G [a,b], which implies that Pr{^ < ^(9,n) and occurs | 
9} > Pr{6 < ^{9,n) and (o occurs | a} for any 9 G [a,b]. So, we have established Pr{6 < 
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^(0, n) and S occurs | a} < Pr{0 < J^{0, n) and occurs | 6} < Pr{a < ^{9, n) and S' occurs | 
6} for any 6 G [a, 6]. In a similar manner, we can show that Pr{& < ^{9, n) and S' occurs | o} < 
Pr{6' < ^{9, n) and (f occurs | 6*} < Pr{a < ^{9, n) and occurs | 6} for any 6 e [a, b]. 

Statement (IV) can be shown by a similar method as that of statement (III). This concludes 
the proof of Theorem [31 

D Proof of Theorem [5] 

It is easy to show that, for Xj € {0, 1}, z = 1, • • • , n, 

PriXi = xi,--- ,Xn = Xn} = h{M,k) where h{M,k) = 

with M = pN and k = X^^Li^^i- Note that h{M,k) = if M is smaller than k or greater 
than N-n + k. For k < M < N - n + k, we have ^^rpTTT = Tldfe^TFFT ^ 1 and only if 
M < ^(A^+1), or equivalently, M < [^(A^+1)J. It can be checked that ^{N+l)-{N-n+k+l) is 
equal to (^ — 1)(A^+1— n), which is negative for k < n. Hence, for k < n, we have that [^(A^+1)J < 
N—n+k and consequently, the maximum of h{M, k) with respect to M E {0, 1, • • • , A^} is achieved 
at \{N + 1)^J . For k = n, we have h{M, k) = h{M, n) = (^^)/(^) , of which the maximum with 
respect to M is attained at M = N. Therefore, for any k £ {0, 1, • • • ,n}, the maximum of 
h{M, k) with respect to M G {0, 1, • • • , iV} is achieved at min {iV, [{N + 1)|J }• It follows that 
min{l, \_^^-^ SILi -^il } is a MLE and also a ULE for p S O. For simplicity of notations, let 
p = min{l, \ {N + 1)|J}- We claim that \p — ^\ < ioi Q < k < n. To prove such claim, we 
investigate two cases. In the case of A; = n, we have p = — = 1. In the case of A; < n, we have 
i^=F[(^^ + l)^J <F(^^ + l)^<^ + ^andp>i[(iV + l)|-l] =^ + ^(^-l)>^-i,. 
The claim is thus proved. In view of this established claim and the fact that the difference between 
any pair of values of p € is no less than we have that — - is a ULE for p G 0. This 
completes the proof of the theorem. 

E Proof of Theorem [7] 

Define /i^ = ^^^^ — - and F£{x) = Pr{/i^ < x, Z = ^} for ^ = 1, • • • ,s, where I is the index of stage 
when the sampling is terminated. Let cr^ denote the variance of X. 
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To show statement (I), note that 

\E[fi-fi]\ < E\il-fi\^Y. \x~f^\dFe{x) 

«_i J — oa 



E / \x-^i\dF,{x)+ f 

1=1 L-^i^-a'Ktsj -^1= 



|a; - /^l dFt{x) 



\x - n\ dFi{x) 



< 



< 



E/ \x-^^\dF,{x)+Y, f 

V/ -^dF,{x)+y^[ ^i\x ~ dFt{x) 

V— / dF,{x)+y^^i \x ^ dFi{x) 

''if 1 



Ev^iE[|/i,-//|2] 



By the assumption that inf^>o ^^^^^ > 1, we have that, there exists a positive number p such that 



2 1 ^1 

(7 1 , v-^ 1 

— = ^ + cr2\ 



< (1 + for all £ > 1. Hence, 



1 a 

< —= + 



rii 
2 °° 



^=1 



^ 1 _1_ 



+ 



a2 1 + p 



as ni —7- oo. Moreover, 
E|/2-/i|2 



E/ k-H'di^K^;) <E^i/^^-^i 

s oo 



1=1 



1 ^a^ {l + pf 

-ni(l + p)2(^-i) nip(2 + p) 



as 77-1 —7- oo. This completes the proof of statement (I). 

Now we shall show statement (II). Since X is a bounded variable, there exists a positive 
number C such that |X — //| < C. By Chebyshev's inequality, we have Pr{|/X£ — n\'> -^j} < 
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for ^ = 1, • • • , s. Therefore, for A: = 1, 2, • • • , 



E 



\x - nl'' dFi{x) + \x - nl'' dFi{x) 



< 



\x~ dFi{x) 
dFe{x) 



< 



E/ \x-^,\^dF,{x) + Y,[ 

yEpr{^ = n + c'=Epr{i/i.-Mi>^} 



as rii — )• oo. Since |E[/i — < — we have that — — )• as rii — )• cxd. This completes 
the proof of statement (II). 

F Proof of Theorem [8] 

We only show the last statement of Theorem [HI Note that 

Us - ni Pr{« = 1} = Us Pr{Z < s} - ni Pr{Z < 1} = ^ {nt Pr{i < £} - n^-i Pr{Z <£-!}) 

s s 

= y (Pr{Z <£}- Pt{1 <£-!}) + '^{ne - ne^i) Pr{l < £ - 1} 

1=2 1=2 

s s 

= y Pr{Z + Y^^ni ~ n,_i) Pr{Z <£-!}, 



e=2 



t=2 



from which we obtain Ug — ne Pr{Z 

that Ug = ni + X^£=2 ('^^ ~ iT'e-i), we have 



S^=2 ('^^ ~ '^^-i) Pr{' < ^ — !}• Observing 



E[n] = ^n,Fi{l=l} = ns- ins-Y,niPr{l = i}\ 

e=i V i=i J 

s s 



e=2 



s~l 



ni+J2 - ni^i) Ft{1 > £ - 1} = m + ^ {ne+i - n^) Pr{Z > £}. 

1=2 1=1 
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G Proof of Theorem [TT] 



To prove Theorem llll we shall only provide the proof of statement (I), since the proof of statement 
(II) is similar. As a consequence of the assumption that f{k + 1) — /(fc) < f{k) — f{k — 1) for 
a < < 6, we have ^^^^^^^^ < f{k + 1) - /(fc) < /(fc) - f{k - 1) < ^^tz^ for a < k < b. Hence, 

m - f{a) ^ M^(^_fc) + /(^(fc-a) 
b — a b — a 

< ^^ib-k) + i^ik-a) m-fja) ^ 
b — a k — a ^ 

which implies f{k) > f{a) + •^^^^^^'-"^ (fc — a) for a < k < b and it follows that 

2^ f{k) >{b-a + l)/(a) + j^—^ 2^{k - a) = . 

k=a k=a 

Again by virtue of the assumption that f{k + 1) — f{k) < f{k) — f{k — 1) for a < k < b, we have 
fik) - f{a) = Y.[f{l + 1) - /(/)] < 5][/(a + 1) - f{a)] = {k - a)[f{a + 1) - /(a)], 

l=a l=a 

6-1 

fik) - f{b) = 5^[/(0 - /(/ + 1)] < - 1) - f{b)] = {k- b)[f{b) - fib - 1)] 

l=k l=k 

for a < k < b. Making use of the above established inequalities, we have 

b i b b 

^/(fc) = (6-a + l)/(a) + j;[/(A:)-/(a)]+ 5] [/(6)-/(a)]+ ^ [/(A;)-/(6)] 

k=a k=a k=i+l k=i+l 

i 

< {b-a + l)/(a) + Y,{k - a)[f{a + 1) - /(a)] 

k=a 

b 

+ {b - i)[f{b) - f{a)] + E - - - 

k=i+l 

= a{i)f{a) + P{i)f{b) 

for a < i < b. Observing that 

■ ^ /(b)-/(a) + (a-b)[/(5)-/(6-l)] _ fe - g - (l - - 
^ /(a + l) + /(6-l)-/(a)-/(6) i + ,(i _ _ 

is the solution of equation /(a) + {i — a)[f{a + 1) — /(a)] = f{b) — {b — i)[f{b) — f{b — 1)] with 
respect to i, we can conclude based on a geometric argument that the minimum gap between the 
lower and upper bounds in ([9]) is achieved at i such that [j\ < i < . This completes the proof 
of Theorem [TTJ 
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H Proof of Theorem [12 



To prove Theorem ll2l we shall only provide the proof of statement (I), since the proof of statement 
(II) is similar. Define g{x) = f{a) + '^^^^1;^^°'' (a; — a) and 



h{x) 



f{a) + /'(a) (x-a) if x < t, 
f{b) + f'{b) (x-b) if X > t 



for t € {a,b). By the assumption that /(x) is concave over [a,b], we have g{x) < /(x) < h{x) for 
X e [a,b] and it follows that j'fix)dx > g{y)dy = and jlf{x)dx < J^^ g{y)dy + 

laiHy) - g{y)]dy with fjh{y) - g{y)]dy = f'^[h{y) - g{y)]dy + - g{y)]dy = Ait). It 

can be shown by differentiation that A{t) attains its minimum at i = ■/(^)~-/(.'^)+°/^(°)~''/ C*) ^ This 
completes the proof of Theorem [T2J 



I Proofs of Theorems for Estimation of Binomial Parameters 
I.l Proof of Theorem I 



We need some preliminary results. The following lemma can be readily derived from Hoeffding's 
inequalities stated in Lemma [TJ 

Lemma 4 S^{k,n,p) < exp(n.^B(f forO < k < np. Similarly, l-SB{k~l,n,p) < cxp(n.^B(^,p)) 
for np < k < n. 

Lemma 5 ^b{z,z — e) < — for < e < z < 1. Similarly, ^b{z,z + e) < — for 
0<z<l-e<l. 

Proof. It can be shown that Q-^bCm+^.m) ^ i^(^l^t^) and ^'^M/^+^.m) ^ i for 

0<e<l-^f<l. Observing that ^b(m,^) = and ^"^''g^e^^^^'' le^o = 0, by Taylor's expansion 
formula, we have that there exists a real number e* G (0, e) such that ^b(m+£7 /^) = t (p+'')(p+E*-i) 
where the right side is seen to be no greater than —2e^. Hence, letting z = fj, + e, we have 
^b(-2) z — e) < — 2e^ for < e < z < 1. This completes the proof of the first statement of the 
lemma. 

Similarly, it can be verified that ^ (^_p_i_ii±£^ and s'-^ni^^-e,^^) ^ 

for < e < ^ < 1. Observing that ^bI/^,/^) = and '^-^bQ^-s^/j) |^^^ _ ^-^y Taylor's expansion 
formula, we have that there exists a real number e* G (0, e) such that ^BifJ-—£, m) = ^ 
where the right side is seen to be no greater than — 2e^. Therefore, letting z = fj. — e, we have 
^b(-2, z + e) < —2e^ for 0<2;<1 — e<l. This completes the proof of the second statement of 
the lemma. □ 



Lemma 6 {Fg {Pi,Pi + e) < C^i Gp {Pi,Pi — e) < (6} is a sure event. 
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Proof. By the definition of sample sizes, we liave > 
—^-^ > — 2e^. By Lemmas 0] and [U we have 



> 



ln(C5) 



and consequently 



Pr{Fp^(p„p, + e)<Cn = Pr{5B(A'.,n„p,+£)<C^} 

> Pr |.^B {Ps3s +e)< > Pr {.^B {p,3s + e) < -2e^] ^ I. 
Pr{Gp, {Pi3i '£)< CS} = Pr{l - (A'. - 1, - e) < (6} 

> Pr |.^B - e) < ^^^} > Pr {^B - e) < '2e^} = 1 

which immediately implies the lemma. 



□ 



Lemma 7 Let < e < |. Then, ^b{z, z + e) > ^b{z, z — e) for z G [O, |] , and ^b(-S) + e) < 
/or 2 G 



Proof. By the definition of the function ^b(-5 Oi '^^ have that ^b(-2, = — oo for z G [0, 1] and 
/U ^ (0, 1). Hence, the lemma is trivially true for 0<z<eorl — e<z<l. It remains to show the 
lemma for z G (e, 1 — e). This can be accomplished by noting that .y#B (-2, 2^ + e) — (-2, 2; — e) =0 
for £ = and that 

d[J^B{z, z + e)- J^b{z, z - £)\ _ 2e^(l - 2z) c 1 _ ^ 

de " (z2-e2)[(i_^)2_£2]' VzG(e,i-ej 

where the partial derivative is seen to be positive for z G (e, ^) and negative for z G 1 — e). □ 
Lemma 8 {^b — || — Ps| )^ ~ |i ~Ps| + ^ '"i^^'' i ^ •'^'^^ event. 

Proof. To show the lemma, it suffices to show .M-q (^^I^^-^Ij^— 1^ — -zI+e) < '"f,''^^ for any 
z G 
ric > 



0,1], since < Pgiyi) < 1 for any w G f^. By the definition of sample sizes, we have 



ln(C-^) 



> and thus > -2e2. Hence, it is sufficient to show J(^{\ - |i - -1 ^ 



•z| + £) < — 2£^ for any z G [0, 1]. This can be accomplished by considering four cases as follows. 

In the case of z = 0, we have ^-q (i ~ | ^ ~ ^| > ^ ~ | i ~ -^j + e) = ^b(0, e) = ln(l— e) < — 2e^, 
where the last inequality follows from the fact that ln(l — x) < — 2x^ for any x G (0, 1). 

In the case of < z < ^, we have ^b (^"II"-^!)^ — — -21+^) = -^b(z, z + e) < — 2e^, 
where the inequality follows from Lemma [5] and the fact that 0<2;<^<1 — e. 

In the case of ^ < z < 1, we have ^b (i ~ |^ ~ -^I ' ^ ~ |i ~ -^I + ^) = -^b(1 — z, 1 — z + e) = 
./#b(-2, z — e')< — 2e^, where the inequality follows from Lemma[5]and the fact that e < ^ < z < 1. 

In the case of z = 1, we have ^b = -^3(0, e) = ln(l— e) < — 2e^. 

The proof of the lemma is thus completed. 

□ 



86 



Lemma 9 {Ji^ (i _ |i , i _ |1 +e) < ^} C {Jl^{%3,^e) < ^b(p„P,- 



s)<'-^}fori = l,- 



. s. 



Proof. Let w e ^ \ ^ ^ Pe \ ^ k ^ \ k ^ Pe \ + < '", ^'''^^ } and pe = Pfl^^)- To show the lemma, 

it suffices to show ma.x{^B {pe, Pi + £), ipe,Pe — e)} < '"^'^'^^ by considering two cases: Case (i) 
Pe < |; Case (ii) pi > i. 

In Case (i), we have ^Bipe,Pe + e) = .-^b ^ \ ^ ^ Pe \ ^ ^ ~ \ ^ ^ Pe \ + £) < '"^^'^^ • Since pe < ^, 
by Lemma El we have ^B(w,Pi! - e) < ^B(w,Pf + e) < 

In Case (ii), we have ^b{pi,Pi-£) = -'^b(1-P£, l-p^ + e) = ^ |i ^ -P^I > 5 ^ ^ P^I + ^ 

i^^^^. Since p£ > ^, by Lemma [71 we have ^B{pe,Pi + s) < J^b.{pi,Pi — e) < ^'^nP • This completes 
the proof of the lemma. 

□ 



Lemma 10 {{\Ps — \\ — — i + 2\n{^S) ^ ^'^''"^ event. 



Proof. By the definition of sample sizes, we have Us > ^-9^ > 3-?-, which implies that 4 + 



< 0. Since " l| " t)^ - is a sure event, it follows that {(|p,,-ihf )2 > i + aWiyi 



21n(C<5) 

is a sure event. This completes the proof of the lemma. 



□ 



Lemma 11 {{\p, - i| - f )2 > 1 + c {^b {Pe,P, + e) < 

£=!,••• ,s. 



7li ' 



{Pi,Pi-e) < for 



Proof Let w e - i| - f )2 > 1 + ^2^} and = Then, 



2ey 1 me^ 
Y) - 4 ^ 21n(C(5)' 



(52) 



To show the lemma, it suffices to show ^ (pe^Pe + s) < ^^n^ ipe^Pe — £) < ^^n^ ■ the 



purpose of proving the first inequality, we need to show 

2 

2 ' 3 y - 4 ' 21n(C(5)' 



1 2e\^ 1 n/e' 
Pi-- + —\>- + 



(53) 



Clearly, ([531) holds if j + j^i^ < 0- I* 

remains to show (j53p under the condition that |: + 2hif^g) > 0- 



Note that (j52|) implies either 



1 













nee'' 



(54) 
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or 



1 













21n(C5)' 



(55) 



Since (j54p implies either 



.2e > 4e. 
3 — 3 



21n(C(5) 



> 



21n(C<5) 



or : 



21n(Ci5) ■ 



it must be true that (|Mj) imphes (|53|) . On the other hand, (|55|) also implies (|53|) because (|55 
implies 



21n(C<5) - -f^^ 2 



4f . Hence, we have established (j53j) . 



In the case of p£ + e > 1, we have ^ {pe,Pe + e) 



-oo < 



have -i<p/'-i + %<l-e-i + %<i and thus ^ 



. In the case of pe + e < 1, we 



^{pi,Pi + £) 



i + f )^ > 0. By virtue of (p]) . 
ln(C5) 



1 I 2e^2 

2 3 y 



< 



Now, we shall show the second inequality ^ {pe,Pi 

2 



e) < 



'"i^^^ ■ To this end, we need to establish 

111 ' 



Pe 



1 2ey ^ 1 



(56) 



based on (I52p . It is obvious that ()56p holds if | 



the condition that i + "''^ 



or 



21n(C<5) 
- i - ¥ > 



4 ' 21n(Ci5) 

> 0. Since (fM|) implies either 



21n(C(J) 

< 0. It remains to show (|56p under 



4 ' 21n(C<5) 2 3 — Y 4 ' 2 ln(Ci5) 

hand, ([55]) also implies ([56]) because ([55]) implies pi 
established ([56]) . 

In the case of p£ — e < 0, we have ^ {p£,Pi — e) = 



3 — 3 V 4 ^ 21n(C5) 

, it must be true that ([54p implies ([56p . On the other 

Hence, we have 



< 



-oo < 



have -i<e-i-^<p£-i-^<l 



of ([56|), 



{pe,Pe 



^ < i and thus ^ - 



< 



' 21n(C<5)' 

^"^^"^^ In the case of ; 



e > 0, we 



2£A2 



_ 2i^2 

3 y 



Hence, - 1} c {^(p„p,+e) < i^Ml , 
e) < for i = l^ - ■ ■ , s. The proof of the lemma is thus completed. 



[Pe - 
ln(C5) 



B(p„p,+£)<i2M), 



) > 0. By virtue 



□ 



Now we are in a position to prove Theorem [131 

If the stopping rule derived from CDFs is used, then {Dg = 1} is a sure event as a result of 
Lemma [H Therefore, the sampling scheme satisfies all the requirements described in Theorem [21 
from which Theorem 1131 immediately follows. 

If the stopping rule derived from Chernoff bounds is used, then {Ds = 1} is a sure event as 
a result of Lemma [S] Note that ^^{z,p) = inf^o E[e*P'^] and that is a ULE of p for 
£ = 1, ■ ■ ■ , s. By virtue of these facts and Lemmas [8] and [H the sampling scheme satisfies all the 
requirements described in Corollary [H from which Theorem 1131 immediately follows. 
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If the stopping rule derived from Massart's inequality is used, then {Dg = 1} is a sure event 
as a result of Lemma [TOl Recall that ^b{z,p) = inft>o e~*^ E[e*P^] and that is a ULE of p for 
i = 1, • • • ,s. By virtue of these facts and Lemmas [10] and \TT\ the sampling scheme satisfies all 
the requirements described in Corollary [H from which Theorem [13] immediately follows. 



1.2 Proof of Theorem | 

Theorem 1141 can be shown by applying Lemmas 1121 and 1131 to be established in the sequel. 
Lemma 12 For i = 1, - ■ ■ ,s — 1, 

HCS) 1 1 I f ^ ^ HCS) \ 



{Di = Q} = S^^^B{Pi,Pe + s) > i^^||j|-^B(P£,P£-e) > 



rii ) 



Proof. To show the lemma, by the definition of Dg, it suffices to show 

{^B (I - +£) < ^} = l'^B{p„P, + e) < i2K£), .^B(p„P,-e) < '-^} 

for £ = 1, - ■ ■ , s — 1. For simplicity of notations, we denote Pi{uj) by p£ for u £ Q. First, we claim 
that - I5 - wl , 5 - 1^ - P^l + < implies ^B{pe,Pi + e) < i^g^i and ^b(w,w - e) < 

isg^. To prove this claim, we need to consider two cases: (i) < ^; (ii) pi > ^. In the case 
of Pi < |, we have ^Bipe,Pe - e) < ^b(w,W + e) = - \ ^ - Pe\ , ^ - || - Pe\ + e) < ^^^Mi, 

where the first inequality follows from Lemma [7] Similarly, in the case of > 5, we have 
^Bipe,Pe + e) < ^B{pe,Pe - e) = .^b(1 -Pf, I - pe + e) = .-#b - \ ^ ~ Pi \ ^ ^ - \ ^ ~ Pe \ + < 
where the first inequality follows from Lemma [7] The claim is thus established. 

Second, we claim that ^Bipe,P£ + £) < ^'^nP ^^'i ^B{pe,Pe — e) < ^'^nP together imply ^b{^ — 
\^ —pe\, ^ - \^ — Pi\+ e) < '"g*^^ . To prove this claim, we need to consider two cases: (i) p£ < ^; (ii) 
pi > h. In the case of < i, we have ^b — l^^wh?" = ■'^Bipe,Pe + ^ ) < '"^'''^^ 



Similarly, in the case of > |, we have ^b ~ \ ^ ~ Pe \ i ^ ^ \ ^ ^ Pe\ + = ^b{^ — p^, 1 — P£ + e) = 
^Bi.Pe,Pe - e) < This establishes our second claim. 

Finally, combining our two established claims leads to {.^3(5 — \^ ~ Pel^ k ^ \k ^ Pel + ^) - 
= {^BiPi,Pi + e)< ii^, ^B{Pe,Pi -e)< ^-^} . This completes the proof of the lemma. 



□ 



Lemma 13 For i = 1, - ■ ■ ,s — 1, 



^B{Pi,Pi + £) > > = {ni z< Ki < niz}, 



Proof. Since ^^^^"/+"^ = In ^1^^}^!^ - for z G (0, 1 - e), it follows that the partial 

derivative ^-^^^^/+"' is equal to for z = z* . The existence and uniqueness of z* can be established 
by verifying that g'-^B(^^'^+'^) ^ [ 1 4. 1 < for any z G (0, 1 - e) and that 



9^b(-z, z + e) 



dz 



l+2£ e a^B(^,^ + £) 

= In -; TT - 1 n < 0, 



l-2£ i-e2 ' 



l + 2s 
= In - — — - 4e > 0. 
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Since M-q(z* ^z* + e) is negative and ng < have that .'^b{z*,z* + e) > ^^7^. On 



ln(l- 



> 



the other hand, by the definition of sample sizes, we have ng > ni 
which imphes \imz-yo -^siz, z + e) < ^^^^p-- Noting that ^b{z,z + e) is monotonicahy increasing 
with respect to z G (0,z*), we can conclude from the intermediate value theorem that there 
exists a unique number z G [0,z*) such that .J^siz, z + e) ~ ^'^^f^ ■ Similarly, due to the facts that 
.■^Biz*,z* +e) > ^"l^P , lim^^i.e .-#3(2^, z + e) = -00 < ^"^^^ and that ^b^z, z + e) is monotonicahy 
decreasing with respect to z S {z* , 1 — e), we can conclude from the intermediate value theorem 
that there exists a unique number z £ {z* ,1 — e) such that ^Biz,z + e) = ^'^^P ■ Therefore, we 
have ^b{z,z + e) > for z G {z,z), and ^Biz,z + e) < for z G [0,z] U [z, 1]. This 

proves that {^BiPe,Pe + e) > ^^^^^ } = {ne z< Kg < ngz}. Noting that ^b + v,^ + v — e) = 
^ — v,^ — V + e) for any v G (O, we have {^BiPe,Pg — e) > ^^7^} = {"^(1 -z)< Kg < 
ng{l — z)}. This completes the proof of the lemma. 

□ 



1.3 Proof of Theorem | 

We need some preliminary results. 

Lemma 14 lim^-j^o X]|=i e""^'^ = for any c > 0. 



Proof. Let c be a positive number. By differentiation, it can be shown that xe~^^ is monotoni- 
cahy increasing with respect to x G (0, i) and monotonically decreasing with respect to x G (i, 00). 
As a consequence of the definition of sample sizes, the smallest sample size ni is no less than ^^"^^^^^ 
and thus is greater than ^ for small enough e > 0. Hence, n£ e~'^^'^ < sni e~^'^^ if e > is 



sufficiently small. Let p = inf 



f>0 



Ct 



1. Observing that s < 1 



ln(l+p) 



< 1 



ln(l+p) 



and ni > 



In(l-e) ' 



we have 



E 



ng e 



< 



In 1 In ^ 
ln(l + P) 



exp 



cln^- 



Me) , l"c^ 
c ln(l + p) 



B{e) 



for small enough e > 0, where A{e) = j^J''^ exp P ^ and B{e) = '"^^ exp j'^'" P 

Noting that lim^^-i^oo xe~^ = and that — > 00 as e -> 0, we have lim^^o ^(s) = 0. Now we 
show that lim£_5.o -B(e) = 0. Using Taylor's expansion formula ln(l+x) = x — ^+o(x^) = x+o(x). 
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we have In = — ln(l — e) = e + ^ + o(e^) = e + o(e) and 
In 



B{e) 



2e2 



s + o{e) 
I + 0(e) + In ^ 



e + o{e) 
-oie) 



cxp ^- 


cln^ 










cxp ^- 


cln^ 

e 




0(e) 




xS[i+°(i)l In 


1 

2e 




; 


e + 


o(e) 



In (1 + I + o(e)) + In -L 
e + o(e) 



exp 



cln^ 



e + ^ + o(e2) 



0(1) 



B*(e) / 1 



where i?*(e) = f-^") ^ . Making a change of variable j; = ^ and using L' Hospital's rule, we 



have 



l + o(l) VC-^ 



X In f 1 + In f 
lini B (e) = lim g^^ = lim . / w 



lim 



1 



cln^^Tx^ 1 



= 0. 



Therefore, < limsup^^o ELi "^t < \ lime_>o ^(e) + x {^-^^ ' x lime^o B*{£) = 0, which 

implies that \im.^^Q Y^\=i e""**^ = 0. This completes the proof of the lemma. 

□ 



Lemma 15 Let he a function of e £ (0, 1) such that 0<a<il}^<b<l. Then, 



+ 



1 - 2^, 



2V^,(1-^,) SVe'Cl-V'e)^ 



+ o(e^), 



+ 



2(1- V.) ' 3 {I -AY 



+ 



1 + eJ 2{l-ij,) 3(1 -V.) 



Proof. Using Taylor's series expansion formula ln(l + x) = x — ^ + ^ + o(x^) for \x\ < 1, we 
have 

^Bii^e,A+<^) = In (^1 + + (1 - i/;^) In (^1 - 

, V-e / e \ ^ ,1 - -06 



20,(1 -Ve) 3 VV'e 



X O ( -g ) + (1 - V-'c) X O 



(1 - 0,)S 



1 - 20e 



27A,(1-V,) 3V2(1-V,)2 
for e < < 1 — e. Since lime_>o j^j^^ = and 



+0(6^) 



lim ■ 

e-5-0 



1+t l-ipe 



- X o 



lim ■ 

e-5.0 
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we have 



1 + e 



= -In + 



In 1 



1 + e 1 - V'e 



1 



, Q, 1 — V'e 



X o 



1 + e 1 - V.'£ 2 V 1 + e 1 - V'e / 3 V 1 + e 1 - 



1 + e 1 - ^Z-, 

2 



1 



3 1 + e 2Vl + e/ 1 - V'. 3 (1 + e)3 (1 - V^)^ 



+ o(e3) 



2e'^ e-^V-'c 1 e V', 



3,/,2 



2(1 -V.) 3 

^2 



e3 2-V', 



2(1 -Ve) 3(l-^,)2 

Since V'e is bounded in [a, we have 



1 - V^e 3 (1 - 

+ o(e3). 



+ o(e3) 



1 + e 



1 + ey 2(1 -^Ae) 3(1-^, 



□ 



Lemma 16 Let < e < 



2 ■ 



Then, there exists a unique number z* G (^, | + e) suc/i i/iai 



(z, z — e) is monotonically increasing with respect to z £ {£,z*) and monotonically decreasing 
with respect to z ^ (z*,l). Similarly, there exists a unique number z* G (| — e, ^) such that 
^siz, z + e) is monotonically increasing with respect to z £ (0,z*) and monotonically decreasing 
with respect to z £ (z* , 1 — e). 



Proof. Note that 



{z,z-e) 



= In 



l-2e 
l+2e 



and its derivative with respect to e equals to 



.2 



> because In 



l~2e 
l+2e 



equals for e = 



which is positive for any positive e less than i. 



Similarly, 



{z,z — e) 



dz 



z = i,+e 



In 



l-2£ 

l+2e 



4e < because In + 4e equals for e = and its deriva- 



tive with respect to e equals to —jz^ which is negative for any positive e less than i. In view of 
the signs of ^ 



{z,z-e) 



at i, i + e and the fact that ^_^|^£:z£) 



z(z-ey^ (l-2)(l-Z+£)^ 



< 



for any z G (£) 1)) we can conclude from the intermediate value theorem that there exists a 



unique number z* G 



1 1 



+ e) such that 



Lz,z-e) 



0, which implies that ^b(-2, z — e) is 



^2,2 I '-Z ■^"^^^ "^^"^ dz 

monotonically increasing with respect to z G (s, z*) and monotonically decreasing with respect to 
z G (z*,l). 

° " ' " ^ < 



To show the second statement of the lemma, note that '^■^^^^^'^+^) 



lni±2£ 



because In r-'-if 



equals for e = and its derivative with respect to e equals to 



2e2 



rB(z,z+e) 



In - 4£ > 



which is negative for any positive e less than ^. Similarly, 
because In — 4e equals for e = and its derivative with respect to e equals to which 



92 



is positive for any positive e less than i. In view of the signs of 



dz 2 



e, ^ and the 



fact that 



^(^^g)2 + (i_z)(i'"_^_e)2 < for any z & (0, 1 - e), we can conclude 
from the intermediate value theorem that there exists a unique number G — e, ^) such that 
d^ii(z,z+e) _ -^j^ich implies that ^■^{z, z + e) is monotonically increasing with respect to 

^ z—z* 

z S (0, z*) and monotonically decreasing with respect to z E (z*, 1 — e). This completes the proof 
of the lemma. 

□ 



Lemma 17 If e is sufficiently small, then the following statements hold true. 

(I): For £ = 1,2, ■ ■ ■ , s — 1, there exists a unique number Z£ G [Oj | — such that ni 



(II) : zi is monotonically increasing with respect to i smaller than s. 

(III) : limg_^o = — — ~; where the limit is taken under the restriction that s — £ is fixed 
with respect to e. 

(IV) For p G (0, i) such that Cj^ = 4p(l - p) and jp > 1, 

lim^^ = A 

£->o e 3 

where £e = ^ — jp- 

(V) : {De = 0} = {zi<pe<l-ze} fori = l,2,-- - ,s-l. 

Proof of Statement (I): By the definition of sample sizes, we have 

„<^M_<„,<(l±^<i±^fl!if + l) (57) 

^B(0,e) ~ 2 2 y2e^ J ^ ' 

for sufficiently small e > 0. By §7}, we have isK^i > .^3(0,6) and 



ln(C(5) 2 f 2 1 \ -2e2 2 /I 1\ 2e' 



ne \l + Ci nj J{^{).-e,\)\^Cx \2 '2) 



2 



Noting that lime_s>o = and lime_j>o — rr = 1: we have ii^^ < Ji-^ (1 _ ^ 1) < q for 

sufficiently small e > 0. In view of the established fact that ^b(0, e) < — < ~ |) 

for small enough e > and the fact that .M^{z, z + e) is monotonically increasing with respect 
to z G (0, ^ — e) as asserted by Lemma [THl invoking the intermediate value theorem, we have 
that there exists a unique number zn G [0, | — e) such that .J^^i^zi, zi + e) = This proves 

Statement (I). 

Proof of Statement (II): Since n£ is monotonically increasing with respect to £ for suffi- 
ciently small e > 0, we have that ^Bize,ze + e) is monotonically increasing with respect to £ if 



93 



e > is sufficiently small . Recalling that ^■q{z, z + e) is monotonically increasing with respect 
to z E (0, ^ — e), we have that zg is monotonically increasing with respect to I. This establishes 
Statement (II). 



Proof of Statement (III): For simplicity of notations, let bi = ^ _£ for £ = 1, 2, • • • , s — 

1. Then, it can be checked that 46^(1 — h^) = C^-i and, by the definition of sample sizes, we have 

e^/[26^(6^ - 1)J Ui 2e^ (6 

for£ = l,2,--- 

We claim that 9 < ze < for 9 G (0,6^) if e > is small enough. To prove this claim, we 
use a contradiction method. Suppose the claim is not true, then there exists a set, denoted by 
Sg, of infinite many values of e such that Z£ < 9 for e E S^. For small enough e £ S/r, we have 
Z£ + £<9 + £<be + e< |. Hence, by ([58]) and the fact that ^b(-z, + e) is monotonically 
increasing with respect to z E (0, ^ — e) as asserted by Lemma [T6l we have 

^ ^B{ze,ze + e) ^ ^B{0,e + e) _ ey[29{l - 9)] + o{e^) _ b^jl - b^) 
^ ' ey[2be{be - 1)] " eV[2be{be - 1)] ey[2be{l - be)] 9{1 -9) + ''^ ^ 

for small enough e E S^, which implies ''gj^Ig)^ < 1, contradicting to the fact that ^g[}lg)^ > 1- By 
([55]) and applying Lemma [T^ based on the established condition that ^ < < ^ for small enough 
e > 0, we have = '^^'^l^^.l^Xf' = 1 + ^^ich implies - = o(l) 

and consequently lime_>o = b^. This proves Statement (III). 

Proof of Statement (IV): 



Since 



Cs-l, In 



and = 4p(l — p), we can write 



from which we have — = o(e), 



2p{l-p) Ini 



£2 



ln(C5) 



-^b(-Z4,^;4 +e) ' 



1 - o(e) = 1 - — < —j^^ < 1 



2p(l-p) In(C^) 



and thus 



2p(l-p)ln(C5) ^ . ^ I 

? _ --^3(^4, ^4 + ^) ^ 1 + (59) 



For 9 E (0,p), we claim that 6* < 24 < ^ provided that £ is sufficiently small. Suppose, to get 
a contradiction, that the claim is not true. Then, there exists a set of infinite many values of e 
such that Z£^ < if e in the set is small enough. For such e < ^ — p, by (|59]l and the monotonicity 
of ^Biz, z + e) with respect to z, we have 

, , -■J^Bizi,,Zi^+s) ^ -^Bi9,9 + e) _ ey[29{l-9)] + o{£^) _ pjl - p) 
+ ey[2p{l - p)] - eV[2p(l - p)] e^/[2p{l - p)] 9{l - 9) ^ ^ 
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for small enough e in the set, which contradicts to the fact that > 1. This proves our claim. 



Since 6 < zi, < \ \s established, by ([59]) and Lemma [TSl we have 



eV[2p(l-p)] 
and consequently. 



e2/[2p(l-p)] 

1 1 2£(l-2z4; 

zi, (1 - 2:4 ) p{l- p) 3zl (1 - 2:4 ) 

Since 9 < Z£^ < ^ for small enough e > 0, by ([60]) . we have 

= p. Noting that ([6( 

(Z4 2e(l-2z4 



1 + o(e) 



it follows that lime_j.o Z£^ = p. Noting that (j60p can be written as 



+ o(e) = 0. (60) 
— ^(T3^ 0(1), from which 



p{l - p)zt,{l - 24) 3z| (1 - 24)^ 



and using the fact that limg^o - 

^4 - 



P 



p E (0, ^), we have 
2p(l-p)(l-2z4 



e 3(24+^-1)24(1 
for small enough e > 0, which implies that limj_>.o ^ ^ 



^4 



+ o(e) = 



-|. This proves Statement (IV). 



Proof of Statement (V) : Note that 



{£>£ = 0} 



1 

' 2 



Pi 



u 





1 




1 


1 _ 




2 




' 2 " 


2-P^ 



In(C^) ^ 1 

2 



= (p^, P£ + e) > ^^^1^. < ^1 U |.^B (P£, P£ - e) > 



ln(C^ 1 



where we have used the fact that ^3(2, 2 + e) = ^b(1 — 2, 1 

V. ^ , ln(C<5) ^ 1 

•^B [Pe, Pe + e)> , Pi < - 

ng 2 

- , ln(C<5) ^ ll fl 

-^B P£, Pi-e)> , Pi> - 

rii 2 



e). We claim that 



1 



< < 1 - 2f 



(61) 
(62) 



for small enough e > 0. 

To prove ([6T]) . let w G {^b (P£, + f^) > Pe < and = p^iuj). Then, ^b(P£, + 



e > 



and Pe < Since 2^ G [0, ^ — e) and >^b (2, z + e) is monotonically increasing with 
respect to 2 G (0, i — e), it must be true that pi > Z£. Otherwise if < ze, then ^b (p^, Pe + £) < 



(24 2£ + e) 



ln(C5) 



, leading to a contradiction. This proves {^b (Pf, + e) > 



, P£ < 



^} ^ {^f < Pf < ^} for small enough e > 0. 

Now let u! G {z£ < p^ < ^} and p£ = p^(a;). Then, Zf < p£ < ^. Invoking Lemma [T6l that 
there exists a unique number z* G (^ — e, ^) such that ^b (2, 2 + e) is monotonically increasing 
with respect to 2 G (0, 2*) and monotonically decreasing with respect to 2 G (2*, 1 — e), we have 

11 



{pe, pi + e) > min <^ ^b (2^?, 2^ + e) , ^b 



2' 2 



(63) 
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Noting that lime_i.o — J'^n^K , \ = 1, we have ^b(^, ^ + e) > '"^^"^^ for ^ < s if e > is small 
enough. By virtue of ([63]) and (-z^, -Zf + e) = ^^^^p-j we have (pe, Pe + > This 
proves {./#B P£ + e) > ^^7^, < 5} 3 {-Zf < < 5} and consequently ([5T|) is established. 

To show ([62]) . let w G {^b {Pe, Pe — > '"^'^'^^ , > ^} and = Pi{uj). Then, ^-Qipi, Pi — 
e) > ^^^^p- and pi > ^. Since 1 — G + e, 1] and {z, z — e) is monotonically decreasing 
with respect to 2; G + e, 1), it must be true that p£ < 1 — z^. Otherwise if > 1 — zi, then 
{pt, P£ — £) ^ — Zi, 1 — Z£ — e) = iz£, zi + e) = ^^l^P , leading to a contradiction. 

This proves {^b(P£, Pe - e) > ^2^, > |} C {i < < 1 - zj. 

Now let w G { ^ < < 1 — z^} and pi = Pi{oj). Then, ^ < < 1 — z^. Invoking Lemma 
[T6]that there exists a unique number z* G (|, ^ + e) such that ^b {z, z — e) is monotonically 
increasing with respect to z G (e, z*) and monotonically decreasing with respect to z G (z*, 1), we 
have 

Pi- e) > min <^ ^b (1 - ze, 1 - Zi - e) , J^b [t,^ \ ■ (64) 



r 2 

Recalling that ^3(5, i ~ ^) = -^b (|, 5 + ^) > ^^^^ smah enough e > 0, using ([Mj) and 
./#b(1 — -2^5 1 — — = -^bC-z^, -Zf + = ^^^^^) have ^b ^i-, Pe — > ^^n^ ■ This proves 
{■/#B (Pf, — ^) > ^^^^^1 Pe > ^} ^ < Pe < ^ ~ Zi} and consequently (f62]l is established. By 
virtue of ([6T]) and (j62]) of the established claim, we have {De = 0} = {z^ < Pf < ^} U < Pf < 
I — Zi} = {ze < pi < I - Zi} for small enough e > 0. This proves Statement (V). 

Lemma 18 Let = s — jp. Then, 

4-1 s 
lim V ne PrlD^ = 1} = 0, lim V Ui PriDi = 0} = (65) 

£=1 i=is+l 

for p G (0, 1). Moreover, lim^^Qrii^ Pr{Di^ = 0} = if Cj^ > 4p(l — p). 



Proof. For simplicity of notations, let bi = lim^^o zg for 1 < £ < s. The proof consists of three 
main steps as follows. 

First, we shall show that ()65p holds for p G (0, By the definition of l^, we have 4p(l —p)> 
Cs_£^_i_i. Making use of the first three statements of Lemma [T71 we have that Zi < P'^^^"-^ < p for 
all £ < ^£ — 1 if e is sufficiently small. By the last statement of Lemma [T7] and using Chernoff 
bounds, we have 

Pr{Df = 1} = Pr{pf < 2c} + FT{pf > 1 - zj < Pr (p^ < P + ^J'-^ I + Pr |pf > 1 - P + ''J--^ 



< exp I ~2ni I 1 1 + exp I -2ne I 

for all £ < — 1 provided that e > is small enough. By the definition of i^, we have 



1 - VI - Cs^i.+i / 1 - \/l - - P) 
= 7^ < 7; = 
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which imphes that and ^?_^pJ^i^j.^ ^j-g positive constants independent of e > 

provided that e > is small enough. Hence, lime_s.o X^fL^^ nePi{Di = 1} = as a result of Lemma 

M 

Similarly, it can be seen from the definition of £^ that 4p(l — p) < Cs^£,~i- Making use of 
the first three statements of Lemma [T71 we have that Zi > > p for + 1 < ^ < s if e is 

sufficiently small. By the last statement of Lemma [T71 and using Chernoff bound, we have 

PT{Di = 0} = Prize < < 1 - z^} < Pr{p^ > ze} < Pr > ^±^^£±i| < cxp (^-2ne ^ ^''^^ 

for + 1 < £ < s provided that e > is small enough. As a consequence of the definition of £e, 
we have that is greater than p and is independent of e > 0. In view of this and the fact that 
Pr{Z)s = 0} = 0, we can apply Lemma[T3]to conclude that linie^o X^fcf^+i P^'i-^^ = 0} = 0. 

Second, we shall show that ()65p holds for p G 1). As a direct consequence of the definition 
of £e, we have 4p(l — p) > Cg-i^+i- Making use of the first three statements of Lemma [T71 we 
have that zi < ^~p+^'-^-^ < 1 - p for all £ < - 1 if e is sufficiently small. By the last statement 
of Lemma [T71 and using Chernoff bounds, we have 

Pr{r», = l} = Prjp, < + Pr{p, > I - z,} < Pr ip, < ^ ~ ^ ^ 1 + Pr (p, > ^ + ^ ' 



< exp ^— 2n£ 1^ ^ — j j + cxp 2n, 

for all i < is — 1 provided that e > is small enough. As a result of the definition of is, we have 
that ft^-i is smaller than 1 — p and is independent of e > 0. Hence, by virtue of Lemma [T4l we 
have lime_^o J2e=i^ Pr{Di = 1} = 0. 

In a similar manner, by the definition of we have 4j>(l — p) < Cg-e^-i- Making use of the 
first three statements of Lemma [T71 we have that Z£ > ^ P+^'e+'i- > i — ^ for + 1 < ^ < s if e is 
sufficiently small. By the last statement of Lemma [17] and using Chernoff bound. 



Pr{£>£ = 0} = Vi{zi <Pt<l-Z£}< Pr{p^ < 1 - z^} 
< Pr < < —— > < exp —2ni 



for Is + 1 < H- < s provided that e > is small enough. Because of the definition of we have 
that 64+1 is greater than 1 — p and is independent of e > 0. Noting that Pr{Z)s = 0} = and 
using Lemma [T^ we have lime^o Yl,\=e.^+i Pr{D£ = 0} = 0. 

Third, we shall show that hm^-^o Pr{-D4 = 0} = for p G (0, 1) such that 4p(l —p) < Cj^. 
For p G (0, ^] such that 4p(l — p) < Cj^, making use of the first three statements of Lemma 



[TTI we have zi^ > ^ > p if e is sufficiently small. By the last statement of Lemma [T71 and using 



P+be, 
2 

Chernoff bound, we have 



Pr{r>,^ = 0} = Pr{z,. < < 1-ZiJ < Pr{p. > z, J < Pr <{ p. > ^-^^ 1 < cxp ( -2n,^ 
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for small enough e > 0. As a consequence of the definition of i^, we have that b^^ is greater than 
p and is independent of e > 0. It follows that lim£_5.o rii^ Pr{Z)^^ = 0} = 0. 

Similarly, for p € 1) such that 4p(l — p) < Cj^, by virtue of the first three statements of 
Lemma \T7\ we have zf, > l^P+h^ > 1-pif e is sufficiently small. By the last statement of Lemma 
[T71 and using Chernoff bound, 

Pr{Df^=0} = Pr{ze,^ <pf^ <1- zej <Pr{pf^ <1~ zej 
< Pr <^ pg < ^ < cxp -2ni^ 



for small enough e > 0. Because of the definition of If,, we have that be^ is greater than 1 — p 
and is independent of e > 0. Hence, lime_^o iT'I^ Pi'i-^^e = 0} = 0. 

□ 

Now we are in a position to prove Theorem [T^l To show lime_>.o | Pr{p G ^} — P\ = 
lime_^o I Pr{p G — P_\ =0, it suffices to show 



lim^Pr{£>£_i = 0, De = l} = 1. (66) 
This is because P < Pr{p e ^} < P and P - £ = J2e=i Pr{-D£-i = 0, De = 1} - 1. Observing 



that 

-1 £,-1 



^ Pr{r>,_i = 0, £>£ = 1} < ^ Pr{r>, = 1} < ^ Pr{r>, = 1}, 

1=1 1=1 t=i 

s s s s 

Pr{r>£-i - 0, r>, = 1} < Pi-{-D£-i = 0} = ^ Pr{r>, = O} < ^ Pr{r>, = 0} 

£=£e+2 £=£e+2 £=£e + l £=£e + l 

and using LemmaUHl we have linie_i.o Y^\l^i Pr{-D£-i =0, = 1} = and linie_i.o X]£=£^+2 Pr{-D£-i = 
0, Di = 1} = 0. Hence, to show ([66|) . it suffices to show lime_j.o[Pr{-D4_i = 0, D^^ = 
1} + Y>v{Di,^ = 0, = 1}] = 1. Noting that 

Pr{£),^_i =0, D,^ = l}+Pr{D4_i = l}+Pr{D,^ =0, = l} + Pr{D,^ = D^^+i - 0} 

= Pr{r>4 = 1} + Pr{Z>4 = 0} = 1, 

we have 

Pr{r>,^_i = 0, Dt^ = l}+Pr{D,, = 0, = 1} = 1-Pr{r>,^_i = D,^ = l}-Pr{D,^ = = 0}. 

As a result of Lemma [THI we have \\m^^^PT{D(^^i ~ = 1} < lime_i.o Pi"{-D^,-i = 1} = and 
lim,^o Pr{£>£, = r>4+i = 0} < lim,^oPr{ = 0} 0. Therefore, lim^^o E£=i Pr{L>^_i = 
0, Di = 1} = 1. This completes the proof of Theorem 1151 



1.4 Proof of Theorem | 

To prove Theorem 1 16| we need some preliminary results. 
Lemma 19 lim,^o jj^^ = ^p, hm,^o ^^^.l^y^^^ = d^^- 
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C ' In 

Proof. By the definition of sample sizes, it can be readily shown that lini£_i.o — ^ 
1 < i < s and it follows that 

^4 



1 for 



= lim 

e-s-O 



lim — : — r-^r = x „ ^ In — 



2p(l-p) 



HC6) 



C. 



2e2 



c. 



3p 



lim 



lim e 



Cs-e 



■In 



2£2~ ~ 4p(l -p) ~ 4p(l -p) 
d 



1 



Cs-P 



^p{l - p)/ne^ e^o y 2e2p(i - p) (5 y 4p(l - p) 



□ 



Lemma 20 Let U and V be independent Gaussian random variables with zero means and unit 
variances. Let = s — jp. Then, for p G (0, ^) U 1) such that Cj^ = 4p(l — p), 



lim Pr{Z = 4} = 1 - lim Pr{Z = 4 + 1} = 1 - $ {vd) , 
e^O e-s>0 

lim [Pr{|p,^ _ p| > £, Z = 4} + Pr{|p,_^+i _ p| > ; = 4 + 1}] 

= Pt {U > d} + Ft {\U + ^V\ >dy^TTp^, U <iyd}. 



Proof. By symmetry, it suffices to show the lemma for p £ (0, ^). For simplicity of notations, 
define ^ 

Ze-P £ rr P£-P 



-p)/ni' 



VpU^^^pT/ne 



Up 



\Jp{\-p)lni 



for = 1, • • • , s. Since Cj^ = 4p(l — p), we have 



np 



2p(l-p) In 



and 



lim I 



lim , = . 

^^o^p{l-p)\ 



2p(l-p)ln^ 



C5 



Hence, by Statement (IV) of Lemma [T71 



lim ( 

£^0 



lim bf^ lim 



ze, -P 



d lim 



^4 -f* 



-vd. 



£-s>0 e-5>0 e e-s>0 £ 3 

Let r/ > 0. Noting that {p^^ < z^} = {Ut^ < a^} and {\pi^ — p\ > e} = {\UiJ > 64}, we have 

for small enough e > 0. Since Ui^ converges in distribution to a Gaussian random variable U 
with zero mean and unit variance as e — t- 0, it must be true that 

Pt{U < -vd-ri} < limPr{p^^ < z^} <Pt{U < -i^d + -q} , 

Pt{\U\ >d + r-i, U < -vd-ri) < limPr{|p^ ~p\>£, Pi < zij < Pr{\U\ >d-r], U < -ud + r]} . 

e— >0 " " 
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Since the above inequalities hold true for arbitrarily small ?? > 0, we have 

limPr{p^^ <Zi^} = Pi{U < -vd} = Y>i {U > vd} = I - ^{ud), (67) 
limPr{|p^ -p\>e,Pi < ze.\ = Vx {\U\> d, U < -ud} = Vi {U > d} . (68) 

Now, we shall consider Prjlp^^^]^ — p\ >£, > zi^}. Note that 
and 

Ui^+i = J—^Ue,_ + Jl- —^Vi^, where Vi^ - 



For small enough e > 0, we have 

Pr{|P4+i -Pl > Pi, > ^4} < Pr{|f^4+il >d-7], Ui^> -vd-r]} , 

-Pl > Pe, > 2:4} > Pr{|f^4+il >d + r], Ui^ > -i^d + r]}. 

Note that Ue, and V^^ converge in distribution respectively to independent Gaussian random 
variables U and V with zero means and unit variances. Since the characteristic function of Ug^+i 
tends to the characteristic function of {U + ^/p^V) / y^l + Pp, we have 



Pi'{\Ue,+i\>d-Tj, Ui^ > -i^d - 7j} ^ Py {\U + ^V\ > (d-7?)yTT^, U>-ud-7j}, 
Pr{|C/4+i| > d + Ue^ > -,^d + ,]}^Pr{\U + ^V\ > {d + 7])^TT^, U>-ud + r^} 

as e — 7- 0. Since r] can be arbitrarily small, we have 

liniPr{|p,^+i-p| >e, p,^ >z,J = Ft{\U + ^V\ > d^l + pp, U > -i^d} 

= Pr {\U + ^V\ > d^/lTp^, U < lyd} (69) 

for p £ (0, ^) such that Cj^ = 4p{l — p). Noting that 



Pr{p,^ < ze^ or p,^ > 1 - > Pi-{1 = 4} > Pr{p,^ < z,^ or p,^ > 1 - z,J - ^ Pr{r>, = 1}, 

e=i 

1,-1 

Pr{l - z,^ > p,^ > ZiJ > Pt{1 = 4 + 1} > Pr{l - 24 > Pi, > Zi,} - Pr{Di,+i = 0} - ^ Pr{I?f = 



and using the result that lime_^o Z^^Li^ Pr{-D£ = 1} + Pr{D£^+i =0} = as asserted by Lemma 
[T8l we have lim^-^oPrl' = ^e} = lhne-5>o Prjp^ < ■^4 or > l — zg^} and limj_^oPi'{^ = + = 
lim£_>oPr{l — -24 > P4 > •24}- We claim that lim£_j.o Pr{P4 > 1 — 24} = for p G (0, |). To 
show this claim, note that limj_^o(l ~ — p) = 1 — 2p > as a result of Statement (III) of 
Lemma [T71 Therefore, 1 — zg^ — p > \ — p for small enough e > 0. By virtue of the Chernoff 
bound, we have Prjp^^ > 1 — ^4} < exp(— 2nf^(^ ~p)^) for small enough e > 0, from which the 
claim immediately follows. This implies that 

lim Pr{Z = 4} = lim Pr{p» < ^£ }, lim Pr{Z = 4 + 1} = lim Pr{p. > z^}. (70) 
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Combining (|67j) and (jTOj) yields 



limPr{Z = 4} = 1 - ^>(z^(i), limPr{i = 4 + 1} = <^{ud). 

e—>-0 £— 5>0 



Noting that 



Pr{|P4"Pl>e, Z = 4}>Pr{|p,^-p|>£, p,^ ^ (z,^ , 1 - z,J} - ^ Pr{r>f = 1}, 
Pr{|P4+i -Pl > i = 4 + 1} >Pr{|p,,+i-p| >e, S (z,^,l-ZfJ} 

-Pr{r>,,H.i=0}- ^Pr{£>, = l} 



and using the result that lime_j.o J2f=i^ Pr{-D^ = 1} + Pi'i-D^^+i =0} =0, we have 
liminf [Pr{|p,^ -p\>e,l=i,}+ Pr{|p,^+i _ p| > £, / = 4 + 1}] 
> lim [Pr{|p^^ -p\>e, p^^ i (z^, 1 - z^)} + Pr{|p^^+i -p| > £ (^4, 1 - . 

On the other hand, 

lim sup [Pr{|p^^ -p| >e,l = 4} + Prjlp^+i - p| > e, Z = 4 + 1}] 
< lini [Pr{|p^^ - p| > e, P4 ^ (^4' 1 - ^4)} + Pr{|p4+i - p| > e, P4 G (^4, 1 - Z4)}] . 



Therefore, 



lim [Pr{|p,^ _ p| > Z = 4} + Pr{|p,^+i _ p| > ; = 4 + 1}] 



lim [Pr{|p^^ -p| > e, ^ (24, 1 - Z4)} + Pr{|p^^+i -p| > e, P4 G (24 > 1 - ^4)}] 



Imi [Pr{|p^^ - p| > e, P4 < ^4} + Pr{|P4+i " Pl > e> P4 > ^^l] • (71) 



Combing ([M]), (ESj) and (IZT]) yields 

lim [Pr{|p,^ _ p| > Z = 4} + Pr{|p,^+i _ p| > Z = 4 + 1}] 
= Vt{V ■>d}^VT{\\J ^ ^^V\>d^[YTp'p.'U <vd) . 

This completes the proof of the lemma. 



□ 



Lemma 21 Let c? > 0, p > and < < 1. Zei J7 and V be independent Gaussian variables 
with zero mean and variance unity. Then, 

2[l-$((i)] <PT{U>d} + PT{\U + y/pV\ > dy/1 + p, U <iyd} ^■^{p,iy,d) + <P{iyd)-<P{d) <3[l-$((i)]. 
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Proof. Clearly, 

Fr{\U + VP^I > dy/l + p, U<vd]< Pr{|[/ + ^V\ > d^l + p] 
= Pr{|C/| > 4 = 2[1 - 

Since > 0, we have 

Vt{\U + ^V\ > + p, U < ud} = Pt{\U + ^V\ > d^\ + p, [/ < 0} 



+ Pr{|^7 + V^^| > dyjl + p, ^<U<ud} 
> Pr{|[/ + ^V\ >d^l + p,U < 0} 

= ^Pr{|[/ + VP^| >dV^} 
= \pi{\U\>d} = l-^d). 



Note that 



Pr{C/ > rf} +Pr{|f/ + > d/r+P, U<vd} 



= Pt{U> d}+PT{U < i^d} - Pr{\U + ^V\ < dy/T+p, U < vd} 



Pr{C/ > rf} +?!■{[/ < vd] - l + Py{\U + ^V\ > d^/TTp or U > vd} 
Pr{C/ > d] -Pr{C/ > vd] + Vt{\U + ^V\ > d^\ + p or U > vd} 



= Pt{\U + y/pV\ > d^/T+p or U>iyd}- Pi{iyd <U <d} 

and that Pr + y/pV\ > d^yl + p or U > vd^ is the probability that {U,V) is included in a 
domain with a boundary which is visible for an observer in the origin and can be represented in 
polar coordinates (r, (j)) as 

(r,</.) : r = — -<j)L < <^< U \{r,4,) : r = ^ ,. , ./-i/ < < 27r - 



COS(p| J 1^ I COS(v^ Y'py 



Hence, by Theorem 6 of [TT] , we can show that Pr { |f/ + ^V\ > d^/TTp or U >ud] = z^, d). 
The lemma follows immediately. 

□ 



1.4.1 Proof of Statement (I) 

First, we shall show that Statement (I) holds for p G (0, ^] such that Cj^ = 4p(l — p). For this 
purpose, we need to show that 

1 < lim sup —J) — ^— < 1 + Pp for any w G | lim p = p \ ■ (72) 

e^O A/a(p,£) l^s^O J 

To show linisupg_^o jsf%\) ^ 1' '^ote that C^^^+i < Ap{l — p) = C^^^ < C^-^^i as a direct 
consequence of the definition of 1^ c^nd the assumption that Cj^ = 4p(l — p). By the first three 
statements of Lemma [T71 we have lim£_5.o Z£ < p for all £ < Noting that lim£_j.op(w) = p < |, 
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we have zi < p{oo) < 1—zg for all i < ^£ — 1 and it follows from the definition of the sampling scheme 
that n(a;) > n^^ if e > is small enough. By Lemma [19] and noting that = 1 if Cj^ = 4p(l —p), 
we have lim sup^^o > linie^o ;v^fe) = = 1. To show limsup^^o < 1 + Pp, we 

shall consider three cases: (i) is = s; (ii) is = s — 1; (iii) ig < s — 1. In the case of is = s, 
it must be true that n{u)) < = n^. Hence, lim supg_j,o < -^™s^o aT^I) — '^p — 

1 = 1 + Pp. In the case of = s — 1, it must be true that n(a;) < Us = n^+i. Therefore, 
limsup.^o < lim,_,o = linie^o ^ x linie^o = = I + Pp- In the case 

is < s — 1, it follows from Lemma [T7] that lime_>.o -s^^+i > p, which implies that z^^+i > 
p, p{u!) < -z^+i, and thus n{uj) < n^^+i for small enough e > 0. Therefore, limsup^_j.Q < 
liirig^o f}'^lpl) ~ l™c^o "n^'^^ X linie_j.o ^ ^-^ = — ^ + Pp- This establishes ([72]) . which implies 

{1 < limsup£_^o j\ffps) — 1 + Pp} — {li™e_>oP = p}- Applying the strong law of large numbers, 
we have 1 > Pr{l < lim sup^^g j^f^ps) — ^ + Pp} ^ {lim£_>o P = p} = 1- This proves that 
Statement (I) holds for p E (0, i] such that Cj^ = 4p(l — p). 

Next, we shall show that Statement (I) for p G (0, ^] such that Cj^ > Ap{l — p). Note that 
Cs-e^+i < 4p(l — p) < Cs„4 as a direct consequence of the definitions of is and jp. By the first 
three statements of Lemma [T71 we have liuis^o Z£^^i < p < ^. It follows that Z£ < p < ^ for 
all £ < — 1 provided that e > is sufficiently small. Therefore, for any uj £ {lime_j.oP = p}, 
we have Z£ < p{uj) < 1 — Z£ for all i < is — I and consequently, n(a;) > rii^ provided that 
e > is sufficiently small. On the other hand, we claim that n{uj) < rii^ provided that e > 
is sufficiently small. Clearly, this claim is true if is = s. In the case of is < s, by the first 
three statements of Lemma [T71 we have linis^oZi^ > p as a consequence of 4p(l — p) < Cg^i^- 
Hence, p{uj) < Z£^ provided that e > is sufficiently small, which implies that the claim is 
also true in the case of is < s. Therefore, n(a;) = n^^ provided that e > is sufficiently 
small. Applying Lemma [T9| we have lime^o j^\p\-) ~ lime^o ^" ^ = Hp, which implies that 
{limg^o jij- (" ^) = Hp} 12 {lime~>o P = p}- It follows from the strong law of large numbers that 
1 > Prjlimg^o ATaTp.e) ^p} - Pr{lims^oP = p} and thus Pr{linie^o ^vl^ = Hp} = 1. Since 
1 < /^p < 1 + Pp, it is obviously true that Pr{l < limsupj_j.Q j^f^ps) ^ 1 + Pp} = 1- This proves 
that Statement (I) holds for p G (0, ^] such that Cj^ > 4p(l — p). 

In a similar manner, we can show that Statement (I) holds for p G 1). This concludes the 
proof of Statement (I) . 

1.4.2 Proof of Statement (II) 

In the sequel, we will consider the asymptotic value of in three steps. First, we shall show 

Statement (II) for p G (0, 1) such that Cj^ = 4p(l — p) and jp > 1. By the definition of the 
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sampling scheme, we have 

E[n] - ^?i,Pr{i = ^}+ nePr{l^e} + ne^Pi-{l^l,} + ne^+iPiil^e, + l} 

1=1 t=lc+2 

< ^ nfPr{D, = 1}+ n,+iPr{D, = 0} + n,^Pr{i =4} + nf^+iPr{i = 4 + 1} 

and E[n] > Pr{Z = 4} + "-f^+i Pr{^ = 4 + !}• Making use of Lemma [18] and the assumption 
that sup£>o ^^^7" < oo, we have 



lim 



,s-l 

£=1 C=f, + 1 



< lim 

e-i-O 



-1 s-1 

ni+1 



^ Pr{D, = 1} + sup ^ ^ 7i,Pr{£), = 0} 



= 0. 



Therefore, 

hmSUp— r- r 

E£S' Pr{^f = 1} + Efcl +1 "^+1 Pr{^f = 0} + n,, Pr{Z = 4} + n^+i Pr{/ = 4 + 1} 
^ ^ TTl ^ 

ne^Pr{l = £,} + ne^+iPT{l=£, + l} 
On the other hand, 

EH n4 Prjf = 4} + n^+i Prjf = 4 + 1} 
hmmf ^ ^ ; V > hm . . , , -■ 

It follows that 

lim = lim ^4 Pr{f = 4} + n^+i Prjf = 4 + 1} 

e^oA/;(p,e) 6^0 A/'a(p,e) 

for p G (0, 1) such that Cj^ = 4p{l—p) and jp > 1. Using Lemma [20l and the result lim£_!.o ^-^ = 
asserted by Lemma [T9l we have 

lim ""^^ ^^^^ ^ ^ -^""^^ =4 + 1} ^ lim "^^^^-^ ~ ^^"""^^^ ^ nt^+i^{vd) 

= 1 + (z^d) . 

Second, we shall show Statement (II) for p G (0, 1) such that Cj^ = 4p(l — p) and jp = 0. In 
this case, it must be true that p = \- By the definition of the sampling scheme, we have 

4-1 4-1 
E[n] = Y,^t = e} + Pr{Z = 4} < ^ Pr{£>^ = 1} + n^^ 

^=1 £=1 

and E[n] > ni^ Pr{Z = 4} > (l - Efc^i^ Pr{i?£ = 1}). Therefore, by Lemma HI 



imsup ; V < hm — ^ = hm ,^ . 



104 



, E[n] , ne(l-ZtiMDi = l}) ^ 
hm mf > hm ^ — = lim ... . = Kp = 1 

and thus lime_^o ~ ^ P ^ (0) 1) such that Cj^ = 4p(l — p) and jp = 0. 

Third, we shah show Statements (II) for p G (0, 1) such that Cj^ > 4p(l — p) . Note that 

E[n] = = i}+ Yl P^^' = ^} + n4 Pr{Z = 4} 

^=1 f=4+l 
4-1 s-l 

< Pr{r>£ = 1} + "^+1 ^""^^^ = 0} + 

f=i £=4 

and E[n] > Pr{Z = 4} > ji^ (^1 - Efc^^ Pr{i?£ = 1} - Pr{D£^ = 0}). Therefore, by LemmalHl 

IE[n] Efc^i' = 1} + Efcl "^+1 Pr{^^ = 0} + n4 
hmsup —r- < hm -r- ^ = hm —r- = 



E[n] "4 (l - Eti' Pr{I), = 1} - Pr{D,, = 0}' 
hmmf —r- > hm -r- = hm ' = k„. 

e->o AAa(p,e) AAa(p,e) £->oA/;(p,e) 



"4 



So, hm£_>o = Kp for p G (0, 1) such that Cj^ > 4p(l — p). From the preceding analysis, we 

have shown hmg^^o fj'^^^) exists for all p € (0, 1). Hence, statement (II) is established by making 
use of this result and the fact that 

y E[n] K{p,e) E[n] 2ln^ E[n] 
hm — ^- = hm —r-, r x hm — ^- = ^ x hm 



e^o7Vf(p,e) 6^0 J\fi{p,e) £^oA/'a(p,e) Z^g e^o A/'a(p, e) " 



1.4.3 Proof of Statement (III) 



As before, we use the notations 4 = , and Ui = ^ 

y/p{l~p)/ne y/p{l-p)/nt 

First, we shall consider p G (0, 1) such that Cj^ > 4p(l — p). Applying Lemma [TSl based on 
the assumption that C^^ > 4p(l —p), we have 

4-1 4-1 
lim Pt{1 < 4} < lim ^ Pr{Dc = 1} < lim ^ ne Pr{De = 1} = 0, 

(=1 e=i 
lim Pi{l > 41 < lim PrjD^ 0} < lim ?7,£ PriD^ = 0} = 

and thus lime^oPr{' / 4} = 0. Note that Pr{|p-p| > e} = Pr{|p^^ - p\ > e, I = 4} + Pr{|p- 
p\ ^ ' 7^ 4} and, as a result of the central limit theorem, Ui^ converges in distribution to a 
standard Gaussian variable U. Hence, 

limPr{|p-p| > e} = limPr{|p„ - p| > e} = limPr{|C/£ \ > be } = Pt{\U\ > d^^} 

and lime^oPr{|p -p| < e} = Pr{|[/| < = 2<^>{d^) - 1 > 2<^>{d) - 1 > 1 - 2(6 for 

p G (0, 1) such that Cj^ > 4p(l - p). 



105 



Second, we shall consider p £ (0,1) such that Cj^ = 4p{l — p) and jp > 1. In this case, 
it is evident that is < s. By the definition of the sampling scheme, we have that Pr{Z > 
4 + 1} < Pr{i?4+i = 0} and that Pt{1 = £} < Pr{De = 1} for £ < 4. As a result of 
Lemma [T8l we have limg^^oPrl' > 4 + 1} < hme_j.o PrjD^+i = 0} = and limg^j^o Prj' < 4} < 
lim,^o Ei=i^ MDe = 1} = 0. Since 

limsupPr{|p — p\ > e} < lim [Pr{|p£^ — p\ > £, I = 4} + ^''^{\Pee+i ~Pl ^ ' = 4 + 1}] 

+ lim Pr{Z < 4} + lim Pr{Z > 4 + 1} 

e-s-O e-!>0 

and liminfe^oPr{|p-p| > £} > lini£->o [Pr{|P£, -p| > e, ' = 4} + Pr{|P£,+i -p\>£, ' = 4 + 1}], we 
have lime^oPr{|p-p| > e} = linwo [Pr{|p,^ > e, i = 4} + Pr{|P£,+i - p| > e, Z - 4 + 1}]- By 
Lemma [20l we have hme^oP^{\p-p\ > e} = Fr {U > d}+Fv{\U + ^V\ > d^l + pp, U < ud} 
for p G (0, 1) such that Cj^ = 4p(l — p) and jp > 1. As a consequence of Lemma \2T\ Statement 
(III) must be true for p G (0, 1) such that Cj^ = 4p(l — p) and jp > 1. 

Third, we shall consider p G (0, 1) such that Cj^ = 4p(l — p) and jp = 0. In this case, 
it must be true that p = ^. Clearly, = s. It follows from the definition of the sampling 
scheme that Pr{Z = £} < Pr{D£ = 1} for i < 4- By Lemma [TSl we have limg^^oPrjZ < 4} < 
lim£_!.o J2e'=i^ F^{D£ = 1} = 0. Therefore, lim£_j.o Pr{Z = 4} = 1 and 

lim Pr{|p — p| > e} ^ lim Pr{|p — p| > e, Z = 4} = lim Pr{|p£^ — p| > e} 

= limPr{|C/£j >biJ^Pr{\U\ > = 2 - 2$(d^) 

for p £ (0, 1) such that Cj^ = 4p(l — p) and jp = 0. 

Note that, for a positive number z and a Gaussian random variable X with zero mean and unit 
variance, it holds true that $(z) = l-Pr{X > z} > l-inf4>o Eie**^-^^^)] = l-inft>o e^*^+V = 
So, (f>(d) = $ (^y^2 In^^ > 1 - ((5 and consequently, liminf£_5.o Pr{|p — p\ < e} > 1 — 2(5. This 
establishes Statement (III). 



106 



1.5 Proof of Theorem [T7] 

Let Ip^ denote the support of for £ = 1, • • • , s. Then, 

s 

m-pi" - \p'-p\'^'^pe^p'^ ^^^^ 



E 



< 



E E \Pi-p\'py{Pe=Pi, i = i}+J2 E iw-pi'pr{p£=P£, ' = n 
Ef^)' E pr{p.=w, / = n+E E pr{p^=pa 



£=1 PfE/p 
\pi~p\>- 



< 



^) ^Pr{i = £} + ^Pr(|p,-p|>^ 



^Pr > 



p \ ^ I p 



for k = 1, 2, • • • , where the last inequahty is derived from Corohary 1 of [8], which asserts that 

-2^ 



Pr -p\>ep}<2 exp -7^ 



hi(l + e) 



l + e 



< 2 exp 



It e 



1, 



for £ G (0, 1). By the assumption that inf, 



> 1, we have that, there exists a positive 



number p such that 7£ < (1 + p)^^^ ^^7i for ah £ > 1. Hence 

\ s /I \ / \ k 

np-p\' < ' ^ 



< 



p 
p 



2E-p(-^/^)^(^) +25:cxp(-l/^(l+pr 



' , 2exp(-iV7r] 







/71/ l-exp(-i^p) 
as 7i — )• 00. Since |E[p — p]| < E|p — p|, we have that E[p — p] — )• as ni — )• 00. This completes 
the proof of the theorem. 



1.6 Proof of Theorem | 

We need some preliminary results. 

Lemma 22 Let < e < 1. Then, ^i{z, is monotonically decreasing with respect to z & (0, 1). 
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Proof. To show that ^i{z, is monotonically decreasing with respect to z G (0, 1), we derive 
the partial derivative as ^^i(z, = jT[ln(l — jq^rj) + iq^rj]) where the right side is negative 
if ln(l — i^^-z ) — i+e~z ■ This condition is seen to be true by virtue of the standard inequahty 
ln(l — x) < —X, Vx G (0, 1) and the fact that < < 1 as a consequence of < 2; < 1. This 

completes the proof of the lemma. 

□ 



Lemma 23 ^i{z, j^) > ^i{z, j^^) for < z < 1 - e < 1. 

Proof. The lemma follows from the facts that ^i{z, j^) = ^i{z, for e = and that 

e 1 d f z \ £ 1 

> TT^I Z 



l + el + e — z de 



1 - £ 



1— £l — £ — Z 



□ 



Lemma 24 {Fp^{pg, < (S, Gp^{pg, < (5} is a sure event. 



Proof. By Lemma HI 



(73) 



> Pr < n,.^B ( ^, j < ln(C<5)| = Pr |3^^b (ps, ) < InCC'J) 



Ps 



l + £ 



< 



7. J 



(74) 



Making use of Lemma [22] and the fact limz^o .^i(z, j^) = — ln(l + e), we have ./#i(z, j^) < 
— ln(l + e) for any z G (0, 1]. Consequently, {^i{pg, r^^) < — lii(l + e)} is a sure event 
because < Ps{u)) < 1 for any u & Q. By the definition of 7^, we have 



ln(l 



> 



HCS) 



ln(l +£)■ 



Since -rf- — ln(l + e) < for any £ G (0, 1), we have ^^^^^ > -rf- — ln(l + e). Hence, 



< 



1 + eJ ~ 1 + e 



ln(l + e) ^ = 1 



(75) 



Combining dZH) and (USD yields PrjGgJp^, j2^) < (S} = I. 
Similarly, by Lemmas and [23l 

Pr [Pp^ (p., < C4 > Pr Isb U,n,, ^) < C^j > Pr |n,.^B ( J^' ) < I^IC^) 



Pr 



Ps 



Ps, 

p. 



<ln(C(5)^ =Prj.^i(p„^ 



< 



1 + e 



< 



HCS) \ 



Is J 

(76) 
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This completes the proof of the lemma. 

□ 

Now we are in a position to prove Theorem [THJ Clearly, is a ULE of p for i = 1, ■ • • , s. 
Define if (p^) = and = for £ = 1, • • • , s. Then, {.if (p^) <Pe< "^{Pi) is a sure event 

for ^ = 1, • • • , s. By the definition of the stopping rule, we have {Di = 1} = (Pi)) < 

C6, Gp^{pi,^{pi)) < C,5} for £ = 1, • • • , s. By Lemma [2^ we have that {Dg = 1} is a sure event. 
So, the sampling scheme satisfies all the requirements described in Theorem [21 from which (|17|) 
and (jlSp of Theorem [18] immediately follows. The other results of Theorem [18] can be shown by 
a similar method as that of the proof of Theorem [19] 

1.7 Proof of Theorem [H] 

Let Xi,X2,--- be a sequence of i.i.d. Bernoulli random variables such that Pr{Xj = 1} = 
l — Y'r{Xi = 0} = p € (0, 1) for i = 1, 2, • • • . Let n be the minimum integer such that XlILi = 1 
where 7 is a positive integer. In the sequel, from Lemmas [25] to [30] we shall be focusing on 
probabilities associated with ^. 

Lemma 25 

Pi- 1^ < z} < exp (cf^i{z,p)) Vz e (0,p), (77) 
Pi- > z| < exp {-f^i{z,p)) Vz € (p, 1). (78) 

Proof. To show ([77]), note that Pr < z} = Pr{n > m} = Pr{Xi + • • • + X„ < 7} = Pr{^%^ < 
^} where m = [^]. Since < z < p, we have < = 7/ [7] < 7/(2) = < we can 
apply Lemma [I] to obtain Pr| ^'=^^^' < ^| < exp (m.^e = exp (7^1 [^,p)). Noting that 

^ < ^ < z < p and that ^i{z,p) is monotonically increasing with respect to z G (0,p) as 
can be seen from ^-^^(^^-p) = J-i^i^^ have < and thus Pr{^ < z} = 

To show dllD, note that Pr > z} = Pr{n < m} = Pr{Xi + • • • + > 7} = Pr |^%^ > 
where m = [^J . We need to consider two cases: (i) m = 7; (ii) m > 7. In the case of m = 7, 
we have Pr > z} = Pr{Xi = 1, i 1, • • • , 7} = HiLi Pi^l^i = 1} = v'' ■ Since (2;,^) is mono- 
tonically decreasing with respect to z G (p, 1) and lim^^^.! {z,p) = Inp, we have Pr > z| = 
p'^ < exp (7^1 {z,p)). In the case of m > 7, we have 1 > ^ = 7/L2J ^ = z > p. Hence, 

applying Lemma [H we obtain Pr| ^'7„'^' > ^} < exp (m^B = exp (7^1 {^,p))- Noting 

that (z,^) is monotonically decreasing with respect to z € {p, 1) and that 1 > ^ > z > p, we 
have (^,p) < ^i(z,p) and thus Pr{^ > z} = Pr{^%^ > ^} < exp (7.^1 (z,p)). 

□ 

The following result, stated as Lemma [26] have recently been established by Mendo and 
Hernando |31j . 
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Lemma 26 Let 7 > 3 and fii > — , Then, Pr{- — > p^i} < 1 — Sp{j — 1, for any 

(0,1). 



Since Pr{^ > {l + e)p} = Pr{2^ > l^{l + e)p} = Vxi'-i^ > pfn} with ^1 = ^{1 + e), we 
can rewrite Lemma [26] as follows: 

Lemma 27 Let < e < 1 and-/> 3. Then, Pr{^ > {l + e)p} < l-5p(7-l, for any p G (0,1) 
provided that 1 + e > — ^ ^ . 

The following result stated as Lemma [25] is due to Mendo and Hernando 



Lemma 28 Let 7 > 3 and ^2 > ^i^- Then, Pr{^ > ^} > 1 - 5'p(7 - 1, (7 - l)^i2) f 



or any 



(0,1). 



Since Pr{^ > (1 - e)p} = Fr{:^ > ^{1 - e)p} = Pr{2-i > ^} with /X2 = Trrrfcr, we 



can rewrite Lemma [28] as follows: 

Lemma 29 Let < e < 1 and j > 3. T/ien, Pr{^ > (l-e)^} > l-5p(7-l, j^) for any p £ (0,1) 
provided that > 1 + . 

Lemma 30 Let < e < I and j £ N. Then, Pr{|^ -p| > ep} < 1 - 5'p(7 - 1, j^) + S'p(7 - 1, y^) 
for any p G (0, 1) provided that 7 > [(l + £ + Vl + 4e + e^) /(2£)]^ + i. 

Proof. For simplicity of notations, let h{e) = [(l + e + VTT^JTe^) /(2£)]^ + i. 

Clearly, Pr 1 1^ — p| > ep} = Pr{^ > (1 + e)p} + 1 — Pi"{^ > (1 — ^)p}- By virtue of Lemmas 
[27] and [29l to prove that Pr {|^ - p| > £p} < 1 - 5p(7 - 1, j^) + 5'p(7 - 1, j^) for any p G (0, 1) 
provided that 7 > h{e), it suffices to prove the following statements: 

(i) 1 + e > , \ , implies ^ > 1 + 

7-2-V7-2 

(ii) 1 + £ > — ^ ^ ^ is equivalent to 7 > h{e); 

(iii) 7 > /i(£) implies 7 > 3. 

To prove statement (i), note that 



1 1 1 , 7 I + V 7 - i 

>1 + — -^£>— -, l+£> ' , -fc^^ P > 



Hence, it suffices to show (^5 + ^7 — ^j/(^7— 5 — ^7 — ?j > ^^^^ , i.e., ^^J, — 2 < y/j. Let 
t = ^. H — \. Then, 7 = + ^ and the inequality becomes 



2- -^^^^") / "12 

2 



7 > ^= - 2 ^t' + ,:>[ - 2 
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i.e., 5t^ - 1*^ - - I > under the condition that - 2 > {t - l)^ > | <^ t> 1 + y^. 
Clearly, 5^^ _ |t2 _ 3^ _ i > 5^3 _ 9^3 _ 3^3 _ 1^3 ^ |t3 > for t > 1 + ,J^. It follows that, for 

t > 1 + y'^, i.e., 7 > 5.4, the inequality holds. It can be checked by hand calculation that it also 
holds for 7 = 1, • • • ,5. Hence, the inequality holds for all 7 > 1. This establishes statement (i). 



To show statement (ii) , we rewrite 1 + e > — ^ ''' , ^ in terms oft=A/7— iasl + e>-j 
which is equivalent to — (1 + e)t — ^ > 0. Solving this inequality yields t > ^+'+^^+''^^+^^ 



7 > h{e). This proves statement (ii). 

To show statement (iii), it is sufficient to show that h{e) > 3 for e G (0,1]. Note that 
h{e) = i[l + .9(e)]^ + 5 with g{e) = (1 + Vl + 4:S + s^)/e. Since g'{e) = -(VTTlFTe^ + 1 + 2e)/ 



2 



(e^Vl + 4£ + < 0, the minimum of h{e) is achieved at e = 1, which is [l + y^j + 5 > 3. 
Hence, 7 > /i(e) implies 7 > 3. This proves statement (iii). 

□ 



Lemma 31 Let = — - where Xi, ■ ■ ■ ,Xn are i.i.d. Poisson random variables with mean 
A > 0. Then, Pr{X„ > z} < exp{n./^p{z, X)) for any z G (A, 00). Similarly, Pr{X„ < z} < 
exp{n^p{z, X)) for any z £ (0, A). 

Proof. Let Y = nXn- Then, y is a Poisson random variable with mean 9 = nX. Let r = nz. If 
z > X, then r > 6 and, by virtue of Chernoff's bound [H], we have 



Pr{X„ >z} = Fi{Y >r} < inf E e*^^-'')] = inf ^ e*^*" 

inf e^^*e-^e-^*y ^e-^^' = inf e-V^^'"^ *, 
t>o ^ i\ t>o 

1=0 



i=0 



where the infimum is achieved at t = In (^) > 0. For this value of t, we have €"^6^*^*"*^ = {t'Y 
Hence, we have Fr{Xn > z} < e~^ = exp(n^p(z. A)). 

Similarly, for any number z G (0, A), we have Pr{X„ < z} < exp{n^p{z, A)). 



□ 



Lemma 32 1 - Sp{j - 1, j^i) + Spij - 1, ^) < 2 [e%l + 

Proof. Let be a Poisson random variable with mean value Let K~ be a Poisson 

random variable with mean value j^. Then, we have Pt{K^ > 7} = 1 — S*? (7 — 1, and 
Pr{ii'~ < 7} = S'p(7 — 1, jz^)- Applying Lemma [31} we have 



Pr{ii'+ > 7} < e%l + e)"(^+^) , Pr{Jf ~ < 7} < 



"(l-e)-(^-^) 



7/(i-s) 
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It follows that 



1 - 7 - 1 



1 + e 



7 - 1, 



1-e 



= Pr{K+ > 7} + Pt{K- < 7} 



< 



e^(l + 



7/{l+e) 



e-(l-e) 



-(l-e) 



7/(l-£) 



< 2 e'(l 



7/(l+e) 



□ 



Lemma 33 For i = 1, - ■ ■ ,s — 1, there exists a unique number Z£ G (0, 1] such that Ji\{zi, 
i^i^ . Moreover, z\> Z2> • • • > Zs-i ■ 



Proof. By the definition of 7^, we have 

ln(C5) 



-ln(l + £) 



< 7^ < 7s = 



ln(C<5) 



which implies < 7£ < ^ '"in(i i ~ • M^a-king use of this inequality and the fact 

lim ( z, ^ I = — ln(l + e) < 0, lim ( z, ^ ] = - ln(l + e) < 0, 

z-^o \ ' 1 + e J 1 + e ^ ' z~^\ \ l + e) ^ ' 



we have 



lim , 



, < < lim z, 

1 + e ) ~ 7^ z^o \^ ' 1 + e 



By Lemma [22l ^i(z, is monotonically decreasing with respect to z G (0,1]- Hence, there 
exists a unique number z^ G (0, 1] such that ^i(zf, -^) = i^^^. 

To show that zi decreases with respect to we introduce function F{z, 7) = ^Jl\{z, j^) — In(C^)- 
Clearly, 



dz 



^F{z,^) 



As can be seen from Lemma [221 and the fact lim^_yo ^liz, j^) < 0, we have ^i{z, < and 
■^^i{z, j^) < for any z G (0, 1]. It follows that ^ is negative and consequently zi > Z2 > ■ ■ ■ > 
Zs-i- The proof of the lemma is thus completed. 

□ 
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Lemma 34 {D(, = 1} C [p^, ^ j < 
Proof. The lemma is a direct consequence of Lemma 



P„^^)<^}/or£=l,...,.. 



□ 



Lemma 35 Ds = 1. 
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Proof. To show Dg = 1, it suffices to sliow ./£i{z, < for any z G (0, 1]. Tliis is because 



{D, = 1} = {^i{Ps, jSfj) < and < p^(a;) < 1 for any w G 0. By tlie definition of sample 



sizes, we have 7^ 



(l+e) ln(C.5) 



> 



(1+5) In(C^) 
-(l+e)ln(l+e) 



Since lim. 



+0 ■ 



-ln(l+e) < 0, we 



e-(l+e)ln(l+e 

have limz_j.o Ji(i{z, < . By Lemma[22l we have that ^i(z, is monotonically decreasing 



with respect to z G (0, 1). Hence, ^i{z, < lini2_j.o < ^"^'''^"^ for any z G (0, 1). Since 



^i(z, is a continuous function with respect to z G (0, 1) and ^i(l, = limz_j.i , 
it must be true that ^i(l, j^) < '"^'•'^^ . This completes the proof of the lemma. 



□ 



Lemma 36 {De = 1} = Pr{p^ > z^} for i = 1, 



,s-l. 



Proof. By Lemma [33l for i = 1, • • • , s — 1, there exists a unique number Z£ G (0, 1] such that 
— l£(C^_ From Lemma [22} we know that ^i{z, j^) is monotonicahy decreasing with 
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respect to z G (0, 1). It follows that < only if z > . This implies that 



, 1^) < = Pi-{Pf > for £ = 1, • • • , s - 1. The lemma is thus proved. 



□ 



Lemma 37 If C, is sufficiently small, then 1 — Sp{'ys — 1, jq^) + Sp{'-fs — l, j^) < ^, inequality 



IS 



satisfied and Pr | 2_p < ej > 1 — 5 /or any p G (0,]5*] . 



Proof. It is obvious that inequality (j22p is satisfied if ^ is sufficiently small. By Lemma [32} we 
have 1 - S'p(7, - 1, j^) + S'pIt. - 1, j^) < 2 [e^(l + e)~^^+''^X"'^^^^\ By the definition of 7^, we 

(l+e)ln(C.5) 



have 7s = 
2[e=(l+e)- 



;-(l+e)ln(l+e) 
_(l+e)l7»/(l+f) 



(l+e)ln(Ca) 
-(l+e)ln(l+e) ' 



which implies 1 - 5p(7s - 1, + S'p(7s - 1, j^) < 
< 2C5. It follows that 1 - 5p(7, - 1, j^) + 5p(7, - 1, < 5 ii C 



is sufficiently small. From now on and throughout the proof of the lemma, we assume that C, is 
small enough to guarantee 1 — Sp (7s — 1, + 5p(7s — 1, jz:^) < S and inequality ([22]) . Applying 
Lemma [36] and (1781) of Lemma [25l we have 



Pr 



< Pr{i = < Pr{Df = 1} = Pr{p^ > < cxp(7f^i(z^p)) (79) 



for < p < Zs-i and £=!,••• ,s — 1. On the other hand, noting that 



Pr 



P 



> e, l = s} = Pi 



> e, i = s ^ < Pr 



> e 



and that 7s > [(l + £ + Vl + -ie + e^) / {2e)] + 5 as a consequence of (p2|) and the definition of 7^ 
we can apply Lemma [501 to obtain 



Pr 



p-p 



P 



> e, 1 = s} <1- Sp[js-l 



1 + e 



Sp [js - 1, 



1 -e 



< 5. 



(80) 



Noting that ^-^^^^^p^ = ^^^^^ > for any p G (0,2:) and that \mip^Q^i{z,p) = -00, we 



have that Yld=\_^^v{li-^i{z(.iP)) decreases monotonically to as p decreases from Zs-i to 0. 
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Since 1 — Sp{'js — 1> j^) + 5'p(7s — yr^) < S, there exists a unique number p* £ (0, z^-i) 
such that 1 — S'p(7s — 1, jq^) + S'p(7s — 1, j^) + Yli\ZX^'^v{li-^i{zi-,P*)) = ^- It follows that 
1 - 5p(7, - 1, + Sp(7, - 1, + E£! exp(7£^i(z^,p*)) < 8 for any p G (0,p*]. Combining 
([79]) and (gOD, wehavePr{|p-p| > ep} < 1-5'p(7s-1, i^) + 5'p(7s-l, + exp(7^^i(^f,p)) < 
J for any p E (0,p*]. This completes the proof of the lemma. □ 



We are now in a position to prove Theorem [191 Clearly, = ^=^^^^ — - is a ULE of p for 
^ = 1, • • • ,8. Moreover, inff>o e~*^ £[6*^^^] = exp(7£^i(2:£,p)) for ^ = 1, • • • , s. Define a random 
interval with lower limit = and upper limit ^{pi) = for £ = 1, • • • , s. Then, 

{^{pi) <P£<'^ (Pi)} is a sure event for £ = 1, • • • ,s. By virtue of these facts and Lemmas [34l 
andl35| we have that the sampling scheme satisfies requirements (i) - (v) described in Corollary [H 
from which (j20p and (j2ip follow immediately. By Lemma [371 there exists a positive number such 
that 1-S'p(75-1, j^)+5'p(7^-l, ^) < 6, inequality ([22]) is satisfied and Pr{|p-p| <ep\p}> 1-S 
for any p E {0,p*] if < C < Co- Hence, by restricting ( to be less than (q, we can guarantee 
Pr{|p — p\ < ep \ p} > I — 5 for any p E (0, 1) by ensuring Pr{|p — p\ < ep \ p} > 1 — 5 for any 
p £ [p* ,1). This completes the proof of Theorem 1191 



1.8 Proof of Theorem [201 

We need some preliminary results. 

Lemma 38 {D, = 1} C (p„ ^) < l^M, (p„ ^) < IbM} £ = i, . . . , 



Proof. For simplicity of notations, define Aii{z,fi) — By tedious computation, we can 

l+e' — 7f 



show that {De = 1} = {Mi{p^, f^) < ^-^} for £ = 1, • • • , s. Noting that 



z \ f z \ 2£^(2 — z) 

YTS) rT^J = 3(1 + 1) [l-^ + .(l-f)] (1-1) [l-z-e(l-l)] 



for < z < 1 — e, we have 

{^,(,,,_|_),!HM,^.(,,,^),!^} 

for 1= 1, • • • , s. This completes the proof of the lemma. 

□ 



Lemma 39 = 1. 
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Proof. To show Ds = 1, it suffices to show M-i{z, j^) < '"^^"^^ for any z G (0, 1]. This is because 



< Psi^^) < 1 for any w G and {Dg = 1} = {Mi{Ps, p^) < ^^r^} as asserted by Lemma 



By the definition of sample sizes, we have 7s ~ 2 (l + |) (1 + £)—^ > 2 (l + |) (1 + ej-^P- 



In ■ 



Since lim^^oMiiz, ^) = -e^ [2 (l + §)(! + e)] ' < 0, we have lini,^oXi(^, ifl) ^ 



Note that Mi{z,yi:^) ~ ^irr. — T\rr~ — r; — tvT) from which it can be seen that M\{z,rr^) is 

J-+e 2(^1+3 j[l+£-(l-3)2j J--l-e 



monotonically decreasing with respect to z G (0, 1). Hence, < \imz^oMi{z, j^) < 

for any z G (0,1). Since Mi{z,j^) is a continuous function with respect to z G (0,1) and 



Is 



Mi{l, = liniz-yi Mi{z, j^), it must be true that Mi{l, j^) < ' . This completes the proof 



of the lemma. 

□ 



Finally, by virtue of the above preliminary results and a similar method as that of Theorem 
[T9l we can establish Theorem 1201 

1.9 Proof of Theorem ED 

Since Pr{n > i} depends only on Xi, ■ ■ ■ ,Xi for all i > 1, we have, by Wald's equation, E[Xi + 
• • • + Xn] = IE[Xj] E[n] = p E[n]. By the definition of the sampling scheme, Xi + • • • + X^ = f, 
and it follows that E[Xi + • • • + X^] = f- Hence, p E[n] = ^[f], leading to the first identity. 

The second identity is shown as follows. Let I be the index of stage when the sampling is 
stopped. Then, setting 70 = 0, we have 

^(7,; - 7,_i) Pr{l >i} = - E T'-i ^ 

i—1 i—1 i—1 

s s—1 s—1 

1=1 j=0 j=0 

s— 1 s 

= 7. Ml > 4 + E = = E ^» P'^i' = = ^[^'] = ^w- 

This completes the proof of Theorem [21] 

1.10 Proof of Theorem [23] 

We need the following lemma. 

Lemma 40 T+e) '^^ monotonically decreasing from to In as z increases from to 1. 

Proof. The lemma can be established by verifying that 

lim ( z, I = 0, lim .-#b \ z, ) = In , lim — ./#b \ | = In \ < 

and [z, ^) - i,-i)(t+e-zr < ^'^y ^ ^ (0' 

□ 
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I.lO.l Proof of Statement (I) 

Let < 7] < I and r = inf£>o By the assumption that r > 1, we have that there exists a 

number i' > max{r, r + ^ + ^^^} such that ^ > for any i > I'. Noting that is 
negative for any ^ > and that 

'■^^^ ^ (£+l-r)ln2-ln(C(5) ^ ^ , 1 \ . . 

'"(C-?^) r + 1 (£-T)ln2-ln(C(5) r + 1 I £ _ ^ - i^i^ / 

\ In 2 / 

for i > £',we have that ^"^^^^^^ is monotonicaUy increasing with respect to i greater than i'. In view 

of such monotonicity and the fact that ^^^^^'^ — —^^7^ — ^ ^ > .#b('7P, jf^) as ^ —t- 00, we have 
that there exists an integer k greater than £' such that ^sivPi j^) < '"^^^'^'^^ for all ^ > k. For £ no 
less than such k, we claim that z < rjp ^b{z, j^) > ^'^'"^^'^ and z € [0, 1]. To prove this claim, 
suppose, to get a contradiction, that z > rjp. Then, since -^^{z,^^) is monotonicaUy decreasing 
with respect to z E (0, 1) as asserted by Lemma HOl we have ^b(z, j^) < .^BivP^ < ^^^177^) 
which is a contradiction. Therefore, we have shown the claim and it follows that {^b(-|j, -jj^y^) > 
C {Kt < vpne} for £ > k. So, 

Pr{« >£}< Pr { ^, 7^^) > < Pr{i^. < vpn,} < exp ^^'^"^ 



for large enough £, where the last inequality is due to the multiplicative Chernoff bound [24 



Since Pr{Z > £} < exp(— ■^^-^^-^^) for sufficiently large £ and — )■ 00 as ^ — )• 00, we have 
Pr{Z < 00} = 1 or equivalently, Pr{n < 00} = 1. This completes the proof of statement (I). 

1.10.2 Proof of Statement (II) 

In the course of proving Statement (I), we have shown that there exists an integer k such that 

znl 
2 



Pr{Z > £} < exp(-c7i£) for any £ > k, where c = 9^ ^ - Note that 



E[n] = ni + ^(n^+i-n£)Pr{Z>£}+ ^ (n^+i-n^) Pr {/>£}. 
e=i £=K+i 

Let R = sup^^o ~n^- Then, n^+i — n£ < Rn£. Hence, if we choose k large enough such that 
cnir'^ > 1, then 

J2 {nt+i - ni,) Pr{Z > £} < ^ (n^+i - n,,) e''^"^ < - ^ cm e"™^ < - ^ crur^ cxp(-cni/) 

f=K+l ^=K+1 £=K+1 l=K. 

^ R r t , i?exp(-cnir-i) 

< — / cnir exp(— cnir = ^ , 

c c liir 



which implies that E[n] < 00. 



1.10.3 Proof of Statement (III) 



By differentiation with respect to e G (0,1), we can show that -^b(-2, 1^7) < ■^b(-Zi i:!:^) for 
< z < 1 - e. It follows that {D^ = 1} = {^b(P£,|^) < ^^^} = {^b{Pi,^,) < 
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we have 



, < s £ = 1, - ■ ■ ,s. Hence, by the definition of the samphng scheme, 



PT{\p,-p\>ep,l = i\p} < Pr<jp<-^, .^Bfp,,-^) <i^^|p 

+ Frip>^,^Jp„^)<'^\p 



1 — £ \ 1 — ni 

< Fr{p<-^,^MP.P)<'-^\P 

I I + e ni 

+ Pr|p>-^,^B(p„p)<i^|p 
I 1 - e m 

< Pr {Gp^ {p„p) < CSe \p}+Ft {Fp^ {p„p) < CSe \ p} 

< 2CSe 

for any p G (0, 1) and £ = 1, 2, • • • . So, ET=i*+i {\Pi - p\ > ep, I = i \ p} < 2C ET=i*+i < 
2(r + 1)C'^, which implies that Pr {|p — p\ < ep \ p} > 1 — 6 provided that ( < 2{t+i) ■ 

1.10.4 Proof of Statement (IV) 

Recah that in the course of proving statement (III), we have shown that Pr — p\ > ep, I = £ \ p} < 
2(6^ for any £ > 0. Making use of such result, we have Pr — p\ > sPi I = £ \ p} 1^ 

2CX]fc=£*+i — V fo'^ ^'^y P ^ (O5 It follows that 

I* 00 
Py{\p-p\>ep\p} = Y,Pr{\pg-p\>ep, l^e\p}+ ^ Pi {\pg - p\ > ep, I = £ \ p} 

t 

< ^Pr{|p,-p| >ep, Z = ^|p} + r/ 
(=1 

e* e 



(=1 1=1 

e t 
< ^^cxp{ne^B{ze,p)) +1] < ^^cxp{ni^B{ze,p*)) + rj < 6 



1=1 



for any p G (0,p*). 

Now we shall bound Pr{p < and Pr{p > for p G [a, 6] C (0,1). Observing that 

{a < C {p > 6} as a consequence of 6 < a{l + e), by statement (III) of Theorem [31 we have 

Pr jb < I <t \ a\ <Vi[p< l<t \ p\< Pr |a < i < T | 
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for any p G [o, b]. On the other hand, 

oo oo 



V 

■Jf ^ — 
- 2 

+1 i=i*+i 



for any p G [a, b]. Therefore, Pr{6 < I < i* \ a} < Pr{p < \ p} = Pr{a < I <t \ 

5} + Pr{p < l>e*\p}< Pr{a < i < T | 6} + 2 for any p G [a, b]. 

Similarly, observing that {b > C {p < a} as a consequence of 5 < a(l + e), by statement 
(IV) of Theorem [3l we have 

Pr |a > l<t \b^ < Pr |p > I < t \ p^ < Pi S^b > I < \ a 

for any p G [a, b]. On the other hand, 

oo 



I 1 — e n/ I 

oo oo 



e=e*+i 

for any p G [a,b]. Therefore, Pr{a > I < £* \ b} < Pr{p > \ p} = Pr{6 > I < £* \ 
a} + Pr{p > I > t \p} < Pr{6 > jl^, i < T | a} + f for any p G [a, b]. This completes the 
proof of statement (IV). 

1.10.5 Proof of Statement (V) 

We need a preliminary result. 



Lemma 41 Let p G (0, 1) and t] £ (0, 1). Let k be an integer greater than niax{r, r + 
such that J^b{vp, T+i) < Then, Pr{l > £} < exp(- ^"^ ) for any £ > k. 

Proof. Let mg = mj^~^ for £ = 1, 2, • • • . Noting that 

^^^^gf^ _ 1 ^ (£ + l-r)ln2-ln(C<5) ^ i ^ (^^ ^ 1 \ < ^ 



ln(C£)^ 
In 2 - 



'"(C^€) J (£-T)ln2-ln(C(5) 7 \ ^ - MC^) 



for £ > max{T, r + ^ + i^^} and that = ^^g^ ^ > ^^ivP, t^) as £ ^ 00, we have 



that there exists an integer k greater than max{r, r + + ^7^^} such that J^Bivp, jf^) < '"^^^^ 
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for all I > K. Since mi < ni and ^■q{'>1P^ v+e^ ^ have that there exists an integer k greater 

than max{T,T + ^ + i^^} such that .^b(?7P, j^) < for all ^>^^. For ^ greater than such 

K, we claim that z < ijp if .^3(2, jj^) > ^^^^7^ and z £ [0, 1]. To prove this claim, suppose, to 
get a contradiction, that z > r/p. Then, since ^b{z, jj^) is monotonically decreasing with respect 
to z G (0,1) as asserted by Lemma HUl we have ^b{z,j^) < ^b(?7P, < ^^^^7^5 which is a 
contradiction. Therefore, we have shown the claim and it follows that |^b(— , ttt^i — ) > iii^^^j c 
{Kf < -qpni} for £ > K. So, 

Pr{Z > £} < P J^B f ^, > ^1 < Pr{A> < ^pna < cxp ^ " ^^'^"^ 



where the last inequality is due to the multiplicative Chernoff bound 



□ 



We are now in position to prove statement (V) of the theorem. Note that 

K 00 

E[n] = ni + Y,i^e+i " ^e) > ^} + Yl (^^^+1 " '^^) ^^^^ > 
e=i e=K+i 

By the definition of ni, we have — ni < (7 — l)n^. By the assumption of e, r/ and k, we have 
In ^ > 1 and thus k > jj^:^ In In ^) + 1 > In (^) + 1, which implies that c'mj'^~^ > 1 and 
^ exp(— cm7'^~^) < e. Hence, by Lemma Wl\ we have 

00 00 00 

^ (n,+i - n,) Pr{Z > < ^ - n,) e"^"^ < ^ cn, e''^"^ 

£=«;+! e=K+l e=K+l 



< -^^ 'S^ cmj^ exp{—c'm^^) < — / cmj^ eyip{—cm"f^)di. 



Making a change of variable x = cm^^, we have d£ = and 
cm7 exp(— cm7 jai = / e ax — 



K—l I J cm-y 



It follows that ET=K+ii^^+i - ^^e) Ml > ^} < ^ '"^^Tn^"''^ < ^exp(-cm7'^-^) < e. This 



completes the proof of statement (V) of Theorem 

I.ll Proof of Theorem [25] 

We need some preliminary results. 

Lemma 42 lim£_5.o X]|=i 7^ e~^^'^ = for any c > 0. 
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Proof. By differentiation, it can be shown that xe~'^'^ is monotonically increasing with respect 
to x S (0, and monotonically decreasing with respect to x E (^,00). Since je > li > in"i+s) 
is greater than ^ for small enough e > 0, we have that e~'^'^ < sji e~^'^^ if e > 



is sufficiently small. Let p = inf. 



1 + 



ln(l+p) 



In 



ln(l+E) 



In 



1. Observing that s < 1 



1 In ( '"(1+^) 

ln(l+p) \^ln(l+e)-^ 



< 



and 71 > 1^7(1^, we have 



E 



li e 



< 



ln(l + p) 



ln(l + e] 



;ln ■ 



for small enough e > 0, where A{e) ~ in{i+l) *^^P 



■ cxp 



-TTO)andi?(£) = 



"ln(l + e) 



c ' ln(l + p) 



In A 



ln(l+£) 



In 



ln(l + e) 



ln(l+£) 



exp 



:: In 



C7 



ln(l+e) ; • 



Noting that lim^_>.oo xe~^ = and that ^^J^-^^ — > od as e 0, we have lim£_s.o ^(s) = 0. Now we 
show that lime_!.o -B(e) = 0. Using Taylor's expansion formula ln(l + x)=x — ^ + ^ + o(x^), 
we have 



ln(l + e) 



e - ^ + oie^) 



ln(l + e)-^ e-f + f + o(e3)-£[l-e + e2 + o(e2)] ^_2|i^^(^3^ 

and 
In ( 



In- 



Inf + m^^ lnf + f + o(e) 



In- 



ln(l + e) 



ln(l + s) 



ln(l 



ln(l 



ln(l 



6 



(81) 



c In 



Using ([5T]) and the observation that [| + o(l)] cxp jj^jiq:^ 

In 2 



= 0(1) + 
= o{l) + 



ln(l + e 
In 2 

e 

£ + o(e) 
In 2 



- cxp 



cxp 



ln(l + e) 



= 0(1) 



— ' = 0(1)) we have 
In^ / 



£ + o(e) 



exp 



cln^ 



£ + o(e) VCfJ 



1 



l + -+o(e) 

-f[l+o(l)] 



0(1) 



B*{e) (I 
1 + 0(1) \C5 



+ o(e2) 



where B*{£) = (i) °. Making a change of variable x = - and using L' Hospital's rule, we 



have 



limB*(e) = lim ^^^^ = lim 



1 + ln(2a;) 



= lim 



0. 



cln^ 



Therefore, < limsup^^o X^Li 7^ e '^'^ < -i linie^o ^(e) + x (^) x linis^o B*(e) = 0, which 



implies that lim£_).o ^£=1 li c ^''^ = 0- This completes the proof of the lemma. 



□ 
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Lemma 43 If e is sufficiently small, then the following statements hold true. 

(I) : For £ = I, - ■ ■ , s — 1, there exists a unique number zg £ (0, 1] such that 7^ = ^^r^^^Vy- 

(II) : zg is monotonically decreasing with respect to i. 

(III) : liuiir^Q zi = 1 — Cs-i, where the limit is taken under the restriction that i — s is fixed 
with respect to e. 

(IV) : For p£ (0,1) such that Cj^ = l-p, 

£^■0 ezg^ 3 

where ie = s — jp. 

(V) :{Dg = 0} = {p,<ze}. 



Proof of Statement (I): By the definition of 7^, we liave 



< — TT < 71 < 7£ < ^ < 5 



^1(1, 



(l + e)ln^ 



C<5 



^l + e)ln(l + e) -e 



+ 1 



(82) 



for sufficiently small e > 0. By ([82]), we have ^-^^ >-^i{l,j^) and 



< 



ln(l 



1 + 



1 

7c 



^-ln(l + e) 2.^1(0,0) 



.^1(0, 0) 



1 + 



ln(l 



1 



(l+£)lii(l+£)-e 



and lim£_j.o 



e-(l+£)ln(l+£) 



1, we have i^Ml < .^j(o,0) 



Noting that lim.^o ^t+^^ - - c... (i+e).^i(o,o) 

for sufficiently small e > 0. In view of the established fact that < < ^i(0,0) 

for small enough e > and the fact that ^i{z, j^) is monotonically decreasing with respect to 
z € (0, 1) as asserted by Lemma [22l invoking the intermediate value theorem, we have that there 
exists a unique number zg £ (0, 1] such that ^i{z£, -f^) = which implies Statement (I). 



Proof of Statement (II): Since is monotonically increasing with respect to i for suf- 
ficiently small e > 0, we have that ^i{zi, 3^) is monotonically increasing with respect to i 
for sufficiently small e > 0. Recalling that ^i{z, j^) is monotonically decreasing with respect 
to z S (0,1), we have that zi is monotonically decreasing with respect to £. This establishes 
Statement (II). 



Proof of Statement (III): For simplicity of notations, let be = 1 — Cs^e for i 
Then, it can be checked that 1 — bi = Cg^i and, by the definition of 7^, we have 



1,2,--- 



[l-6,)(l + e)^i(z,,^) 1 C,„,(l + e)ln^ 



C<5 



e- (l + e)ln(l + e) 



(l + e) ln(l + e)-e 



1 + 0(1) 



(83) 



for 1,2,--- ,s-l. 

We claim that zg < 9 for 9 G (bg, 1) if e > is small enough. To prove this claim, we use a 
contradiction method. Suppose the claim is not true, then there exists a set, denoted by Sg, of 
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infinitely many values of e such that zi > 6 for e G Sg. By (j83p and the fact that ^i{z, j^) is 
monotonically decreasing with respect to z G (0, 1) as asserted by Lemma [22} we have 



1+0(1) = : > 



1 - 



o(l) 



e - (1 + e) lii(l + e) " e - (1 + e) ln(l + e) 

for small enough e £ S^, which implies < 1, contradicting to the fact that > 1. The 
claim is thus established. Similarly, we can show that Z£ > 6' for 0' G (0, bi) if e is small enough. 
Now we restrict e to be small enough so that 9' < Z£ < 9. Applying Lemma [15] based on such 
restriction, we have 



(1 _ 5,)(i + e)^^^ze, if^) _ (1 - [-W^ + o(^')J _ t^I + o{l) 



e - (1 + e) ln(l + e) 



Combining ^ and ([MI) yields 
(III). 



o(l), which implies lim£_j.o - 



(84) 



l + o(l) ■ 
b£. This proves Statement 



Proof of Statement (IV): 

Since 



C-f, (l+£)ln(C.5) 
£-(!+£) ln(l+e) 



and C, 



1 — p, we can write 



jl-p) (l + £)ln(C.5) 
e-{l + e) ln(l + e) 



ln(C5) 



from which we have = o{e) 



(1-p) (l+£)ln(C^) 

l-o(e) = l- — < ^-^^tt'rr^^ <1 



and thus 



(l-p) (l+£) ln(C^) 
£^(l+£)ln(l+e) 



l + o(e). 



For G (p, 1), we claim that ze^ < if e is sufficiently small. Suppose, to get a contradiction that 
the claim is not true. Then, there exists a set of infinitely many values of e such that > 9 ii e 
in the set is small enough. For such e, by the monotonicity of ■/#!(., .), we have 



1 + 0(e) 



(l-p)(l+£)ln^ 
(l+e)ln(l+£)-£ 



> 



(l + e)ln(l+£) 

{l-p){l + e)^i{9,9/{l+e)) _ 1-p 
e- (l + e)ln(l + e) ^1-9 



(85) 



0(1) 



for small enough e in the set, which contradicts to the fact that > 1. This proves the claim. 
Similarly, we can show that > 9' for any 9' G (0,p). Now we restrict e to be small enough so 
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that 9' < Z£^ < 9. By virtue of such restriction, we have 



(1 - + e)^i{ze^,zej{l + e)) 
e- (l + e)ln(l + e) 



e/(l + e)-ln(l + e) 



+ 0(8^) 



e[l - e + £2 + o(e2)] _ _ ^ + ^ + o(e3)] 



£3(2-^ 



■2(T^ + 3(1-.J^ 



-^ + ^+0(^3) 
1-p 2£(l-p)(2-^,J . 



1 - f + o(e) 



(86) 



Combining ([85]) and §El yields ji:^ - '"^aVi-^r)^'"^ = 1 - f + ^(e)' 



l-p _ 2e(1-p)(2-zj!J _ ]^ _ 4e 

4e 2e(l-p)(2-Z4) 



1 



3(1 



+ o(e), 



I.e. 



p-ze^ 4(1 -ze J 2(l-p)(2-Z4) 



3z4(l - 24) 



+ 0(1), 



which imphes that hm£_s.o ^7^7^ = '^^3^^^ — ^^3^^^ 



Proof of Statement (V): Noting that ^i{z,j^-) is monotonically decreasing with respect 



to z e (0, 1) as asserted by Lemma [2^ we have {Di = 0} = > -^^} = {Pe < zg] as 

claimed by statement (V). 



Lemma 44 Let ie = s — jp. Then, 



4-1 



lim J^7^Pr{£)^ = l}=0, lini ^ 7^ Pr{£>^ = 0} = 



e-5>0 



(87) 



.+1 



for p G (0, 1). Moreover, lim^^oT^ Prj-D^ = 0} = if Cj^ > 1 — p. 



Proof. For simplicity of notations, let be = lime_>o zg for 1 < ^ < s. The proof consists of two 
main steps as follows. 

First, we shall show that (j87|) holds for any p € (0,1). By the definition of i^, we have 
1 — p > Cg-i^^i- Making use of the first three statements of Lemma HSj we have ze > p+^^-s-'^ > p 
for all £ < — 1 if e is sufficiently small. By the last statement of Lemma 03] and using Lemma 
we have 



Pr{Di = 1} = Pr{p^ > Zi} <Pr{pi> ^^^^^ ^ [> < exp 



-,P 
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for all £ < ^£ — 1 if e > is sufficiently small. Since 64-1 is greater than p and is independent of e > 
as a consequence of the definition of ie, it follows from Lemma [42] that lime_i.o X^^Li^ leP^'iDe — 
1} = 0. 

Similarly, it can be seen from the definition of that l—p< Cs~e^~i- Making use of the first 



three statements of Lemma H51 we have that < - — |^ < p for + 1 < ^ < s if e is sufficiently 
small. By the last statement of Lemma 03] and using Lemma [25] we have 



Pr{D, = 0} = Pr{p, < z,} < Pr |p, < ^-^^ j < exp (^7^^! (^ ^ ^ ,P 

for ^£ + l<^<sife>Ois small enough. By virtue of the definition of is, we have that 

is smaller than p and is independent of e > 0. In view of this and the fact that Pr{Ds = 0} = 0, 

we can use Lemma [42] to conclude that lime_j.o J2e=i^+i H Pr{D^ = 0} = 0. 

Next, we shall show that lim£_>.o 74 Pr{Di^ = 0} = for p G (0, 1) such that Cj^ > 1 — p. 
Note that 1 — p < Cg-e^ because of the definition of i^- Making use of the first three statements 
of Lemma [l3] we have that z^^ < E+hs^ < p if e > is small enough. By the last statement of 
Lemma [l3] and using Lemma [25] we have 

Pr{D,^ = 0} = Pr{p,^ < < Pr < ^±^| < exp (^74 --^l (^^^p)) 

for small enough e > 0. By virtue of the definition of i^, we have that 64 is smaller than p and 
is independent of e > 0. It follows that lim^^o 74 = 0} = 0. This completes the proof of 

the lemma. 

□ 

Finally, we would like to note that Theorem 1251 can be shown by employing Lemma [44] and a 
similar argument as the proof of Theorem [151 



1.12 Proof of Theorem | 

We need some preliminary results. 

Lemma 45 lim^^o t^j^ = i^p, "^^"^^e-^Q ^ \J'^ = 

Proof. By the definition of 7^, we have 

Cs-i (l + e)ln(C<5) 



lim 



£^0 7£[e - (1 + e)ln(l + 

for 1 < I < s. It follows that 

74 V ^^liP^^e) C,_4 (l + e)ln(C5) C,_4 (l+e)^i(p,^^ 

iim , = lim — - — 7—^ — X — — = hm 



>0 7(p,e) e->o \n{C5) e - (1 + e) ln(l + e) e-^o e - (1 + e) ln(l + e) 

(1 + e) (eV[2(p - 1)] + o{e^)) _ _ C,^ 



e^-o e - (1 + e) ln(l + e) 1-p 1-p 
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and 



lim £ 



lie 



/ 1 C4(l+£)ln ^ 
J™ ^ V (1 + e) ln(l + e) 



C<5 



s-4 



□ 



Lemma 46 Let U and V be independent Gaussian random variables with zero means and unit 
variances. Then, for p £ (0,1) such that Cj^ = 1 — p, 

lim Fr{l = 4} = 1 - lim Fr{l = 4 + 1} = 1 - $ (ud) , 
lim [Pr{|p,^ -p\>ep,l = 4} + Pr{|p,^^+i - p\ > ep, I = 4 + 1}] 

= Pr{C/>(i} + Pr{|C/ + ^F| >dy^TTp^, U < ud} . 



Proof. By Statement (V) of Lemma 031 we have 

e=i 

MPi^ < > Ml = 4 + 1} > Pr{P4 < - Pr{r>^,+i = 0} - ^ Pr{D, = 1}. 



Making use of this result and the fact that lim£_>o 
asserted by Lemma SH we have 



MDe = 1} + Pr{£>4+i = 0} 



as 



lim Pr{Z = 4} = lim Pr{p. > ZiA, lim Pr{Z = 4 + 1} = lim Pr{p. < zgA. 
£->0 e-s>0 ^ £-)-0 e-s>0 

Noting that 

4-1 



4-1 



Pr{|P^,+i - p| > ep, Z = 4 + 1} > Pr{|P4+i - p| > ep, p,^ < - Pr{D,,+i = 0} - ^ Pr{r», = 1} 



and using the result that lime_>.o 



PT{De = 1} + Pr{£>4+i = 0} 



0, we have 



liminf [Pr{|p^^ - p\ > ep, I = 4} + Pr{|p^^+i - p\ > ep, I = ie + 1}] 

On the other hand, 

lim sup [Pr{|p£^ - p\ > ep, 1 = 4} + Pr{|p£^+i - p\ > ep, Z = 4 + 1}] 

e-!>0 

- [^^{\Pie -P\'^ Pie ^ + Pr{|p4 + i - p| > ep, < Z4}] . 
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Therefore, 



lim [Pr{|p^^ -p\>sp, 1= 4} + Pr{|P4+i - p\ > ep, I = £e + 1}] 



lim [Pr{|p^^ -p\> ep, p^^ > z^} + Pr{|p^^+i - p\ > ep, < z^}] 



£^0 



Since = 1, by Lemma [451 and Statement (IV) of Lemma 143} we have 



lim 



P- ze, / 



lim / ^^^^ lim ^ 



1 — p £-s>o y 1 — p e^o eze 



d lim - — — 

e^o ezi 



-—d = —I'd. 
3 



Note that 



1 ne^ 



1 



74 



P 74+1 74+1 V 74 



1 -Prr , 74+1 - 74 
-<^4 + 



1 — p 



74+1 V ^"^(74+1 - 74) 



^4 



where 



/ P^74 



n^+i - 1 \ / p'^{-fi^+i - 74 



.P4 Pj\'^-P " V 74+1 -74 V 

By the central limit theorem, Ui^ — t- U and V^^ — t- V as e — t- 0. Hence, 



+1 



1 1\ p^-fi 



1-p 



74 I^-Ptt , 74+1-74 



V P 7^. 



74+1 



l-p 



74 



Pp 



-V 



74+1 ' V 74+1 ' \j ^ + Pp \ ^ + Pp 
ase^O. It can be seen that Pr{p^^ > z^} = Pr{f/4 < ^^y'^}, 

= Pr{P4+i > (1 + Pf, < 2:4} + Pr{Pf,+i < (1 - Pf. < ^4} 

fl 1 e 1 1 p-ze ] fl 1 e llp-z^ 
= Pr<- <-Tz ^, >- ^^+Pr<^^, > -, >^ ^ 



e 



Pi,+i P (1 - Pi, P pzi. 



(l + e)V l-p 



74+1 rr ^ P - ^4 / 



l-p 



(1-£)V l-p 



-Pr^C/4+i>7T^J?^,t/4>^^^- 



l-p 



and 



Pr{|P£. - Pl > ep, K ^ ^4} = Pi- I'^f. < - 

+ Pr jc/^, > 



(1 + e) V l-p 



74 jj < P- ^4 / 7£. 



l-p 



11. ^P- zi. I 74 



(l-e)V l-p' '^^ - ^4 V l-p. 
Therefore, for p G (0, 1) such that Cj^ = 1 — p, we have lim£_),oPr{Z = 4} = 1 — lim£_^oPr{' 
4 + 1} = 1 - $ (ud) and 

lim [Pr{|p^^ - p| > ep, Z = 4} + Pr{|p^^+i - p| > ep, Z = 4 + 1}] 
Fr{\U\ >d,U< -i^d} + Fr{\U + ^V\ > d^\ + pp, C/ > -z^(i} 
= Pr{?7>4 + Pr{|;7 + ^y| >(iyrT^, C/<i'd} 
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as e — 7- 0. This completes the proof of the lemma. 

□ 

1.12.1 Proof of Statement (I) 

First, we shall show that Statement (I) holds for p £ (0, 1) such that Cj^ = 1 — p. For this purpose, 
we need to show that 

1 < lim sup '^}^\ <1 + Pp for any w e | lim p = p\ . (88) 

To show limsupg_^Q ^^j^ > 1, note that Cg-e^+i < 1— P = Cg-e^ < Cg-e^-i as a direct consequence 
of the definition of and the assumption that Cj^ = 1 —p. By the first three statements of Lemma 
[l3l we have lim£_i.o -s;^ > p for all £<££ — !. Noting that linie^oPi'^) = P, we have p{uj) < zi 
for all £ < — 1 and it follows from the definition of the sampling scheme that 7(t<;) > 
if e > is small enough. By Lemma US] and noting that Kp = 1 if C^^ = 1 — p, we have 
limsup.^o > lim^^o = Kp = 1. 

To show limsupg^Q -^^^^ < 1 + Ppj we shall consider two cases: (i) ie = s — 1; (ii) ie < s — 1. 
In the case of = s — 1, it must be true that 7(1:1;) < 7^ = 74+1. Hence, limsup£_^Q -^^^ < 
lim£_!.o = liniE-s^o ^Jpe) ^li™g_i.o '^l^^^ = In the case of < s—1, it follows from the first 

three statements of Lemma |l3] that lim£_5.o -z^+i < P, which implies that 2:4+1 < p, p(w) > 24+1, 
and thus 7(0;) < 74+1 for small enough e > 0. Therefore, limsupg_j.Q ^^^^ < linie^o '^lp%] = 1 + 
This establishes (f88l) and it follows that {1 < limsupg_j,Q ^(pg) < 1 + Pp} 5 {linie^.oP = p}- 
According to the strong law of large numbers, we have 1 > Pr{l < limsup^^j^o ■y(p£) — ^ + Pp} — 
Pr {lim£_s.oP = p} = 1. This proves that Statement (I) holds for p G (0, 1) such that Cj^ = 1 — p. 

Next, we shall show that Statement (I) holds for p G (0, 1) such that Cj^ > 1 — p. Note 
that Cg-i^+i < 1 — p < Cs-i^ as a direct consequence of the definition of £s and the assumption 
that Cjp > 1 — p. By the first three statements of Lemma |43l we have lim£_>.o 24-1 > p and 
thus Zi > p for all £ < £e — 1 provided that e > is sufficiently small. Therefore, for any 
UJ G {lime_s.oP = p}: we have p{uj) < zg for all £ < — 1 and consequently, 7((^) > 7^^ provided 
that e > is sufficiently small. On the other hand, we claim that 7(t<;) < 7^^. Such claim can 
be justified by investigating two cases. In the case of £e = s, it is trivially true that 7(c<;) < 7^^. 
In the case of < s, we have p > lime_j.o 2:4 and thus p > ze^ provided that e > is sufficiently 
small. Therefore, for any u G {lime_j.oP = p}, we have p{uj) > Z4 and consequently, 7(0;) < 7^^ 
provided that e > is sufficiently small. This proves the claim and it follows that 7(t<;) = 7^^ 
if e > is small enough. Applying Lemma l45| we have limg_>o ^ij^l^ ~ liine_>o jJp'e) = which 
implies that {linie_i.o ^{p e) = ^p} — {lime_^oP = p}- It follows from the strong law of large numbers 
that 1 > Pr{lime^o = Hp} > Pr{linie^o P = p} = 1 and thus Prjlim^^o = Kp} = 1. Since 

^ ^ Kp < 1 + Pp, we have that Pr{l < limsup^^j^Q ^^.^^^ < 1 + Pp} = 1 is of course true. This 
proves that Statement (I) also holds for p G (0, 1) such that Cj^ > 1 — p. The proof of Statement 
(I) is thus completed. 
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1.12.2 Proof of Statement (III) 

First, we shall consider p G (0, 1) such that Cj^ = 1 — p. In this case, it is evident that ie < s. It 
follows from Lemma and the definition of the sampling scheme that lim£_j.oPr{Z > + 1} < 
lim^^o Pr{£>4+1 = 0} = and lim^^o Ml < 4} < hnie^o EI=i^ = 1} = 0. Since 

limsupPr{|p — p\ > ep} < lim [Pr{|p^^ — p\ > ep, I = 4} + Pt{\p^^^i — p\ > ep, I = le + 1}] 



+ lim Pr{Z < 4} + lim Pr{Z > 4 + 1} 

e->0 e->0 



and 



liminf Pr{|p — p\ > ep} > lini [Pr{|p^^ — p\ > £p, I = 4} + Pi'ilP^+i ~ P\ > £P) ^ = 4 + 1}] > 
we have 

lim Pr{|p -p\> ep} = lini [Pr{|p,^ ~ p\ > ep, I = 4} + Pr{|P£,+i - p\ > ep, I = ie + 1}] . 
By Lemma lini we have 



limPr{|p-p| > ep} = Pr{U > d} + Fr{\U + ^V\ > d./TTp^, U < ud} 

for p G (0, 1) such that Cj^ = 1 — p. As a consequence of LemmaEH Statement (III) must be true 
for p £ (0, 1) such that Cj^ = 1 — p. 

Next, we shall consider p G (0, 1) such that Cj^ > 1 — p. Note that Cs-£^+i < I — p < 
Cs-i^- Since Ug^ = (^^^ — 1^ \l converges in distribution to a standard Gaussian variable U, 

liirie^o e^Jj^ ~ and limj_>>o Pr{7 = 7^^} = 1 as can be seen from Statement (I), we have 

limPr{|p — p| > ep} = limPr{|pf^ ~ p| > ep} 



lim FriUf > ^— ■ / I + lim Pr \Ui_ < ^ ' ^'''^ 



e-i-o 1^ ^ l-eyl-pj £^0 [ ' l + eyl-p 



= limPr||C/,J>£^^|=Pr{|C/|>dV^} 

and consequently, lime_>o Pr{|p — p\ < ep} > 2<I> (dy^) — 1 > 1 — 2(^6 for p G (0, 1) such that 
> 1 — p. This proves Statement (III) . 

Finally, we would like to note that Statement (II) can be shown by employing Lemma and 
similar argument as the proof of Statement (II) of Theorem [TBI 



1.13 Proof of Theorem | 

We need some preliminary results. 

Lemma 47 lim^-^o Yle=i n£e~"'^'^ for any c > 0. 

Lemma [17] can be shown by a similar method as that of Lemma [TH 
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Lemma 48 If e is sufficiently small, then the following statements hold true. 

(I): For £ = 1, - ■ ■ , r, there exists a unique number zg S [0, 1] such that un — — '"^^^"^^^ 



(II): zi is monotonically decreasing with respect to i no greater than r. 

-1 



(III) : lim£_>o-2^ = 1 + (1 — ^)Ct-_£ for 1 < £ < r, where the limit is taken under the 
restriction that I — t is fixed with respect to e. 

(IV) : {D, = 0} = {p, <z,] for I =!,■■■ , r. 



Proof of Statement (I): By the definition of n^, we have < < 



Jn(C5) 



ni < 7i£ for sufficiently small e > 0. Hence, lnj^^ = ^b(I) i^) — < small enough 

e > 0. Recall that ^b(-z, is monotonically decreasing with respect to z G (0, 1) as asserted by 
Lemma Hni Invoking the intermediate value theorem, we have that there exists a unique number 
zg G (0, 1] such that M^{zi, = which implies Statement (I). 

Proof of Statement (II): Since is monotonically increasing with respect to i for sufH- 
ciently small e > 0, we have that ^b(-2^5 i+e) monotonically increasing with respect to £ < r 
for sufficiently small e > 0. Recalling that ^b(-2j t^) is monotonically decreasing with respect 
to z € (0, 1) as asserted by Lemma [40l we have that zn is monotonically decreasing with respect 
to £ < T. This establishes Statement (II). 



Proof of Statement (III): 

For simplicity of notations, let hi = 1 + (1 — ^)Cr- 



-1 

for £ = 1, 2, • • • , r. Then, it can be 



checked that \^J'J}^ = Cr~e for 1 < i < t. By the definition of sample sizes, we have 



^B(..^) _ln(C.)^2(,--l)a„.^^^^^^^ ^^^^ 



e%e/[2{h - 1)] ni p*e^ 
for ^ = 1, • • • , T, where 

ln(C<5) [l + o(l)]a_,ln(C5) 



Hi 



We claim that < z^ < \ for 6 G (0, h^) if e > is small enough. To prove this claim, we use a 
contradiction method. Suppose the claim is not true, then there exists a set, denoted by S^, of 
infinite many values of e such that Zi < 9 for e £ S^- Hence, by (j89p and the fact that ^b(-2, jj^) 
is monotonically decreasing with respect to z G (0, 1) as asserted by Lemma HOI we have 

. , ^b{z,,^J J^^{e.^) eH/[2{l-e)] + o{e^) Ojl-h) 

^ ^ ^ e%e/[2{be - 1)] " e^eMbe - 1)] e^be/[2{l - be)] 6,(1 - 9) ^ > 

for small enough e £ Ss, which implies > 1, contradicting to the fact that ^^^^^"J'^j < 1. 

This proves our claim. In a similar manner, we can show that < Z£ < 9' for 9' G (6^, 1) if 
e > is small enough. By ([89]) and applying Lemma [15] based on the established condition that 
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e <Zi<e' for small enough e > 0, we have f.'^/y^l^^^l^ = ^'^^^B^a-M] = 1 + «(!)' ^^ich 
implies — = o(l) and consequently \\m^^Q zn = bi. This proves Statement (III). 

Proof of Statement (IV): Noting that ^siz, j^) is monotonically decreasing with respect 
to z e (0, 1) as asserted by LemmagOl we have {De = 0} = {.J^siPi, y^) > ^-^^^} = {Pe < ^e} as 
claimed by statement (IV). 

Lemma 49 Let ie = t ~ jp- Then, 

4-1 T 
lim V Pr{i:>^ = 1} = 0, lim V niPi:{Di = 0} = (90) 

£-S>0 £-5-0 

i=l i=is + l 

for p £ {p* ,1). Moreover, lim£_5.o n^^ Pt{D£^ = 0} = for p £ (p* , 1) such that Cj^ > r{p). 

Proof. For simplicity of notations, let bi = lime_>.o Z£ for 1 < i < t. 

First, we shall show that (j90p holds for p £ (p* , 1). By the definition of i^, we have r(p) > 
Cr-e^+i- Making use of the first three statements of Lemma HH} we have that > P+^^f-^ > p for 
all £ < — 1 if e is sufficiently small. By the last statement of Lemma 08] and using Chernoff 
bound, we have 

Pr{D, = 1} ^ Pr{p, > .a < Pr [p, > } < cxp (^^2n, (^^^ 

for all £ < £e — 1 provided that e > is small enough. By the definition of we have 



^1 



>P, 



which implies that is a positive constant independent of e > provided that e > is 

small enough. Hence, lime^o X^fLi^ Pr{-D£ = 1} = as a result of Lemma W7\ 

Similarly, it can be seen from the definition of ig that r{p) < Cr-i^-i- Making use of the first 
three statements of Lemma HH] we have that Zi < ^'^''^^'^^ < p for + 1 < ^ < r if e is sufficiently 
small. By the last statement of Lemma 08] and using Chernoff bound, we have 

Pr{D, = 0} = Pr{p, < z,} < Pr < P±^| < exp (^-2ne ' 

for + 1 < ^ < T provided that e > is small enough. As a consequence of the definition of ^g, 
we have that 64+1 is smaller than p and is independent of e > 0. Therefore, we can apply Lemma 
07] to conclude that linie_i.o X^fc^+i Pr{-D£ = 0} = 0. 

Second, we shall show that lim£_>.on^g PrjD^ = 0} = for p G {p* , 1) such that Cj^ > r{p). 
Clearly, r{p) < Cr-i^ because of the definition of i^- Making use of the first three statements of 
Lemma HS] we have zg^ < Elhh. < p if e is sufficiently small. By the last statement of Lemma 08] 
and using Chernoff bound, we have 

Pr{r>,^ = 0} = Pr{p,, <zej< Pr \p, < ^^^\ < cxp ( -2ne^ ' 
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for small enough e > 0. As a consequence of the definition of i^, we have that bi^ is smaller than 
p and is independent of e > 0. It follows that lim£_)>o n^^ Pr{Z)£^ = 0} = 0. 

□ 



Lemma 50 linie^o S^^r+i = i} = for any p £ {p*,l). 

Proof. Recalling that the sample sizes ni,n2, • • • are chosen as the ascending arrangement of 
all distinct elements of the set defined by (p5]) . we have that 



p* \ 



for small enough e G (0, 1). By the assumption that infjgz -ft^ = 1 + p > 1, we have that 



ni> {1 + p) 



ln(C<5) 



i = T+l,T + 2,' 



for small enough e G (0, 1). So, we have shown that there exists a number e* G (0, 1) such that 



P , 



l + e 



< (l + pr-"^Mn(C5), e = T + l,T + 2, 



for any e G (0,e*). Observing that there exist a positive integer k* such that {1 + pY~'^~^ ln(C'5) < 
ln((^5) — — r)ln2 = ln{(^5e) for any i > t + k* , we have that there exists a positive integer 
K* independent of e such that ^sip*,-^) < ^^^^^ for £ > r + k* and < e < e*. Recall that 
■^b{z, j^) is monotonically decreasing with respect to z G (0, 1) as asserted by Lemma HOl For 
i>T + K* and < e < £*, as a result of ^^^^^ > ^b(p*, y^) > ./^b(1, j^) = In j^, there exists a 
unique number zi G [0, 1] such that ^B{ze, = ^^^^7^ > ^b(p*, i^ji^)- Moreover, it must be true 
that Zi < p* for i > t + k* and e G (0,e*). Therefore, for small enough e G (0,e*), we have 



r+K. 



^ 7i<. Pr{« = e} ^ X! ^'''t^ = ^} + XI P^'i^ = ^J' 

£=T+1 i=T+K*+l 

< X nePT{Dr=0}+ riePi-{Di_i=0} 

i=T+l i=T + K- + l 

= Y nePT{Dr^O}+ J2 ne+iPi-{Di=0} 

i=T+l i=T + K' 

00 

< fc*(l + p)'^'*n^Pr{Dr = 0} + (1 + p) Y nePr{De^O} 

00 

< k*{l+p)'''nr PT{Pr <Zr} + {l+p) ^ Pr{pf < 



< k*{l + p)'''nrcxp(-^{p- p*f\ + {1 + p) Y neexp{-2ne{p- p 



-T-\-K* 
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as e — )• 0, where we have used Chernoff bound and the assumption that supjg^ "U^ = 1 + p < oo. 
This completes the proof of the lemma. □ 



Lemma 51 lim^ 



ep 



Proof. By the definition of sample sizes, it can be readily shown that lime^o ^''^ ^-.j'TJL ^ = 1 
for any i > I and it follows that 



lim 



oM(p,e) 



^.^.^B(.,^)^2(i-p-)a_.. mi. 



>o ln(C(5) 

-2 



lim 



2(1 -P) 



2(i-p*)a-4 



iin.^^_ = 



e-i-O 



^/pil-p)/ne^ y ]3(l-p)p*£2 y P*(l-J3) 



□ 



Lemma 52 Lei U and V be independent Gaussian random variables with zero means and unit 
variances. Then, for p G (p*, 1) such that Cj^ = r{p), 

lim Pr{l = 4} = 1 - lim Pt{1 = 4 + 1} = 1 - $ (ud) , 
£-s>0 e-s>0 

lim [Pr{|p£^ -p\>£p^ 1 = 4} + Pr{|p^^+i - p\ > ep, I = is + 1}] 



= Pr{f/>4 + Pr{|C/+^y| >(iv/rTp;;, f/ < z^d) . 
Lemma [52] can be shown by a similar method as that of Lemma HHl 

1.13.1 Proof of Statement (I) 

First, we shall show that Statement (I) holds for p £ {p* , 1) such that Cj^ — r{p). For this purpose, 
we need to show that 

1 < lim sup "f^^ < 1 + Pn for any a; G | lim p ^ p \ ■ (91) 

To show lim sup^-^g A^lp,''^) - wte that Ct-_£^+i < r{p) = Cr-i^ < C^_4_i as a direct 
consequence of the definitions of and jp. By the first three statements of Lemma 148} we 
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have lim^^Q Zi > p for all £ < £e — 1. Noting that lim£_j.op(a;) = p, we have p(c<j) < 
for all ^ < — 1 and it follows from the definition of the sampling scheme that ne^ < n(a;) 
if e > is small enough. By Lemma [51] and noting that Kp = 1 ii Cj^ = r{p), we have 

limsup.^o ^ f&) '^P 1- 

To show limsup^_^Q < I + pp, note that + 1 < r as a result of p* < p < 1 and the 

assumption that Cj^ = r{p). By virtue of Lemma SHI we have lime_j>o < which implies 

p{uj) > zg^+i and thus n{uj) < for small enough e E (0, 1). Therefore, limsupg_j.o fj^''^\) < 

lime_j.o J}'Ip_1^ = liine-i-o "'ni^^ ^ Ume^o jj^{p.e) ^ ^ + Pp- This establishes ([9T]) , which implies { 1 < 

limsup£_j.o ^ ^" < 1 + Pp} 5 {lim^^j^oP = p}- Applying the strong law of large numbers, we have 

1 > Pr{l < limsupj_j.o (" g) < 1 + Pp} ^ Pi"{liiii£_s>oP = p} = 1- This proves that Statement (I) 

holds for p G (p*, 1) such that ~r{p). 

Next, we shall show that Statement (I) holds for p € (p*, 1) such that Cj^ > r{p). Note that 
Cr-e^+i < r{p) < Ct-_4 as a direct consequence of the definition of is and the assumption that 

> r{p). By the first three statements of Lemma BHl we have lim£_!.o zi^^i > p > lime_j.o z^^ and 
thus ze > p > Z£^ for all i < — 1 provided that e G (0, 1) is sufficiently small. Therefore, for 
any uj G {lim£„j.oP = p}^ we have Zi > p{uj) > Z£^ for all £ < — 1 and consequently, n{u) = ni^ 
provided that e G (0, 1) is sufficiently small. Applying Lemma [51] we have lime^o = 
linie^o _/v'"(p ~ which implies that {limg_).o ^ ^" = ^p} I) {limj^oP = p}- It follows from 
the strong law of large numbers that 1 > Prjlime^o ^ (" ~ i^p} > Pr{lime^op = p} and thus 
Pr{lini^_,o jr;^ = Kp} = 1- Since 1 < Kp < l+Pp, we have that Pr{l < limsup^^g A?7§7) - ^+Pp} 
is of course true. This proves that Statement (I) holds for p £ (p*, 1) such that > r{p). The 
proof of Statement (I) is thus completed. 

1.13.2 Proof of Statement (II) 

In the sequel, we will consider the asymptotic value of in three steps. First, we shall show 

Statement (II) for p G (p*,l) such that Cj^ = r{p). Clearly, < r. By the definition of the 
sampling scheme, we have 

— 1 r oo 

E[n] = ^ n£ Pt{1 = i} + ^ ne Pt{1 = i} + ^ m Pt{1 = i} 

1=1 f=«e+2 £=T+1 

+n,^ Pi{l = 4} + n,^+i Pi{l = 4 + 1} 

le-l T-1 oo 

< ^n^Pr{Df = 1}+ ^ ne+iPr{De = 0}+ ^ nfPr{«=^} 

i=i £=i^+i e=T+i 

+ne^ Pt{1 = 4} + ne^+i Pt{1 = 4 + 1} 
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and E[n] > n^^ Fi{l = + rt£^+i Pr{Z = ie + !}• Making use of Lemmas 1491 1501 and the assumption 
that sup^^o < °o for small enough e > 0, we have 



lim 

e-J-O 



nePT{De = 1} + ^ Pr{r>, = 0} + ^ n,Pr{«=^} 



< lim 



Therefore, 



and 



It follows that 



.1 e=ie+l £=T + 1 

i^-l T-1 OO 

^ ?i£Pr{£)^ = l} + sup^^ niPr{De^O}+ ^ nePT{l ^ £} 

e=i ""^ i=T+i 



lim sup -77 < lim -r- 

E[n] n,^ Pr{« = 4} + r^£.+i Pr{« = 4 + 1} 
limmt — r- > lim -r- . 

lim ^["] = lim ^^^^ = + "^-+1 = 4 + 1} 



Using Lemma [52] and the result lim^^j^o J^h \ = Kp as asserted by Lemma [511 we have 



lim ""^^ '^''^^ ^ ^ ""^^^^ '^'^^^ =4 + 1} ^ lim "^-'^"^ ~ ^^"""^^^ ^ n4+i^(;^(i) 

= 1 + (l/d) . 

Second, we shall show Statement (II) for p G (p*, 1) such that Cj > r{p). Note that 



4-1 



E[n] = ^ Pr{Z = £} + ^ Pr{Z = ^} + Pr{Z = 4} + ^ Pr{Z 

£=1 e=ie + l i=T+l 

4-1 T-1 00 

< ^?i^Pr{£>^ = l} + ^n^+iPr{£)^ = 0}+n4+ J]] n^Pr{Z=£} 



=r+l 



and E[n] > n^^ Pr{l ^ 4} > ne^ (l - Efci^ Pr{Df = 1} - Pr{D£^ = 0}). Therefore, by Lemma[49l 



E[n] j:ti^riiFi{De = l} + j:lJni+iPT:{De = 0}+ni^+j:Zr+iniMl^f} 

hm sup — r- < lim -r- r 

e^o Kip,s) e^o Kip,e) 
= lim —r- = Kp, 

E[n] ^4 (1 - Et'i P^{Di = 1} - Pr{£>4 = 0}) „^ 
limmf -7- > hm ^ — = hm = Kp. 

So, hm^^o = P ^ (f'*' 1) '^^'^ t'^^t > r(p). 

From the preceding analysis, we have obtained limsupj,_^o jj'^{^e) ^^'^ ^ ^ {P* A)- Hence, 
statement (II) is established by making use of this result and the fact that 

E[n] Mr{p,e) E[n] 21n^ E[n] 
lim . ' \ = lim —T-^, X lim . , , \ = x lim 



£-5>o A/f(p, e) £-s>o7Vf(p, e) e->o7Vr(p, e) 2^^^ e->oAAi.(p, e 
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1.13.3 Proof of Statement (III) 

First, we shall consider p G (p*, 1) such that Cj^ = r{p). In this case, it is evident that < r. By 
the definition of the sampling scheme, we have that Pr{Z > + 1} < Prji^^^+i = 0} and that 
Pt{1 = £} < Pr{De = 1} for £ < 4- As a result of Lemma l49| we have lime^oPrjZ > 4 + 1} < 
lim^^o Pr{£>4+1 = 0} = and lim.^o < 4} < lim.^o Ee=i^ MDe = 1} = 0. Since 

limsupPr{|p — p\ > ep} < lim [Pr{|p^^ — p\ > sp, I = 4} + Pr{|p£^^;^ — p\ > ep, I = + 1}] 



+ lim Pr{Z < 4} + lim Pr{Z > 4 + 1} 

e^O e-s>0 



and 



liminf Pr{|p — p\ > ep} > lini [Pr{|p^^ — p\ > ep, I = 4} + PrUp^^.,.]^ — p| > ep, Z = 4 + 1}] , 
we have 

lim Pr{|p -p\> ep} ^ lim [Pr{|p,^ ^ p\ > ep, I ^ 4} + M\Pi^+i -p\>ep,l=le + 1}] ■ 

By LemmaESl we have lime^o Pr{|p-p| > ep} = Pr{C/ > d} + {\U + ^V\ > dy/l + pp, U < vd} 
for p e {p* , 1) such that Cj^ = r{p). As a consequence of Lemma EH Statement (III) must be 
true for p G {p* A) such that Cj^ = r{p). 

Next, we shall consider p E {p* , 1) such that Cj^ > r{p). Applying Lemma HH we have 

lim Pt{1 < 4} < lim PT{De = 1} < lim ^ ne Pr{De = 1} = 0, 

i=i 1=1 
lim Pt{1 > 4} < lim Pr{J[)f^ = 0} < lim Pi{De^ = 0} = 



and thus lim^^oPrlZ / 4} = 0. Note that Pr{|p — p\ > ep} = Pr{|p^^ — p\ > ep, I 
41 + Prjlp — p\ > ep, I 7^ 41 and, as a result of the central limit theorem, Ue = , ^ 
converges in distribution to a standard Gaussian variable U . Hence, 



lim Pr{|p -p\> ep} = lim Pr{|pf - p| > ep} = lim Pr < |;7f, | > I = Pr{|L/| > dyK^} 

and lime_^oPi-{|p - p| < ep} = Pr{|C/| < = 2^{d^) - 1 > 2^{d) - I > I - 2Q5 for 



p G (p*, !)• Here we have used the fact that $(z) > 1 - e" V and ^{d) = <i>(y 2 In ^) > 1 - (,5. This 
proves Statement (HI). 



1.14 Proof of Theorem | 

We need some preliminary results. 

Lemma 53 ./£b{z,z — e) is monotonically increasing with respect to z £ {e,p + e) provided that 
< e < II and <p < ^ ~ ife. 
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Proof. Define g{e,p) 

that g{£,p) >OifO<e<|| and < p < i - ^fe. 



p(i-p) 

35 
94 

Let i < A; < 1 and < e < 



In (^+.)fi-'') for < p < 1 and < e < 1 - p. We shall first show 



2(ij^k) • '^^'^ ^® shown by tedious computation that '^^(^■^2^ 



(i_4fc:je^)^ [ i-4(fc-i)^£^ ] ' which implies that g (e, | — fee) is monotonically increasing with respect to 



16e^ [3fc-l-4(l-fe)fc^e^] 

^ ^ ('^i 2l\/r^ ^^"-^ monotonically decreasing with respect to e e {^■^\J jz^ ~ 3, 2(iq 
Since g (O, ^) = 0, we have that g {e,\ — ke) is positive for < e < ^(i^fc) if 9 (^7 h ^ 
positive for e = 2{i+k) ■ ^ ~ 2{i+k) with A; = we have g (e, | — fee) = 1 + tttA-t — In (2 



(i+fc) 



IS 



2fc+l 



1 + II - In (2 + yI), which is positive because e x esg > 2.718 x X^j^o \W 



1 ^35^* > 2 + f|. It follows 



that 5 (e, i - i|e) is positive for any e £ (O, ||) . Since ^^4^ = 



12 



9a{e,p) _ 
dp 

negative, we have that g{e,p) is positive for < e < || if < p < | 

dz^ ~ ^ z{z~ey^ 



1 



Finally, the lemma is established by verifying that 



for any z G (e, 1) and that 



d.4^B {z,z — s) 
dz 



z=p+e 



(l-p-e)(l-p)^ 



IS 



< 



□ 



Lemma 54 ^b(p — £,p) < 



+ e,p) < -2e^ forO<e<p<^<l-e. 



Proof. The lemma follows from the facts that ^b{p — £,p) — ^Bip + £,p) = for e = and 
that 



b {p-£,p) - ^b{p + £,p)] 
de 



In 



1 + 



2p- 1 



p2 (1 — p)2 — 



where the right side is negative for 0<e<p<^<l — e. By Lemma O we have ^b{p + ^,p) < 
— 2e^ for 0<e<p<^<l — £. This completes the proof of the lemma. 

□ 



Lemma 55 ^Bi^, jr^) is monotonically decreasing from to — oo as z increases from to 1 — e. 
Proof. The lemma can be shown by verifying that 



lim I z , 



= 0, lim . 

z— >-l — e 



1 -£ 



-OO, lim -^^B ( z, 

z-s-o dz \ 1—e 



= In- 



< 



o2 

and -S-^ 

dz-' 



iz-i)(i-e-z\i < any z G (0, 1 - e) 



□ 



Lemma 56 ^b{z, 



> 



) /or < z < 1 - e < 1. 
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Proof. The lemma follows from the facts that 



l+e^ 



z, j^) = for e = and that 



d_ 



1 - e 



2e'^z{2 - z) 



(l-e2)[(l_z)2_e2 



> 



for z G (0, 1 - e). 



□ 



Lemma 57 {^b < < ^} 



is a sure event. 



Proof. For simplicity of notations, we denote p* = in order to show the lemma, it suffices 
to show 

P.-r^)>'^,Ps>P^-ea]=^. (92) 

I- ErJ Us J 

3s + ea) > P,<P*-ea]=9, (93) 

1 + / J 

„p,-e,) > i^^, <p^ + ej =0. (95) 
n 



By the definition of n^, we have > ^ /"ii*^^ — ^ > ^ /"^yj — tt- By the assumption on Sa 



^B(P*+£a,P*) 



ln(C-?) 



3(P*+£a,P*) ' 



and Er, we have < Sa < p* < < 1 — Ea- Hence, by Lemma [Ml we have {p* — £a,P*) < 
.^B {p* + £a,P*) < and it foUows that 

ln(C5) 



>^B(p* + ea,P*) >^B(p*-ea,P*). (96) 



By m, 



(97) 

Noting that ^b (p* — £a,P*) = -^B (^P* — ^a, ^le" ^ making use of the fact that „#b(-z, jr^) 
is monotonically decreasing with respect to z G (0, 1 — e) as asserted by Lemma \55\ we have 

(ps, T^) > ip" - ea,p')\ = {Ps <P*- ea}. (98) 



1 - , 

Combining ([97]) and §^ yields 1^. Bv (f96]l . 

.^b(Ps,Ps + ea) > ii^^, < - £a| C {y£B{Ps3s + '^a) > -^B (p* - £a,P*) , < - Ea} ■ 



(99) 

By the assumption on Ea and Er , we have p* — Ea < ^ — £a- Recalling the fact that ^b {z,z + £) is 
monotonically increasing with respect to z E (0, ^ — e) as asserted by Lemma [T6l we have that the 
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event in the right-hand side of (j99p is an impossible event and consequently, (j93p is established. 
By m, 



(100) 

Noting that (p* + Ea,P*) = -^B (^P* + ^a, ^ and making use of the fact that ^b(-z, j^) 
is monotonically decreasing with respect to z G (0, 1) as asserted by Lemma l40l we have 

{^B (ps, yI^) > (P* + ea,P^)} = {p. <P'' + £a}. (101) 

Combining (fTnO]) and (fTUT]) yields ([SID. Bv (fMD . 



^b(Ps:Ps - Ea) > , < P* +£a ^ C {^b(Ps,Ps " £a) > [p* +ea,P*) , < +£«}• 

(102) 

By the assumption on Sa and Er, we have that ^Biz,z — e) is monotonically increasing with 
respect to z € {£a,P* + £a) as a result of Lemma [53l Hence, the event in the right-hand side of 
(I102p is an impossible event and consequently, (I95p is established. This completes the proof of 
the lemma. 

□ 



Now we are in a position to prove Theorem [28l If the multistage sampling scheme follows a 
stopping rule derived from Chernoff bounds, then {D^ = 1} is a sure event as a result of Lemma 
EZl Note that ^b(^,p) = inft>o e"*^ E[e^Pe] and that is a ULE of p for ^ = 1, • • • ,s. So, the 
sampling scheme satisfies all the requirements described in Corollary [H from which Theorem [28] 
immediately follows. 

If the multistage sampling scheme follows a stopping rule derived from CDFs, then, by Lemmas 
U we have 

1 > Pr{Gg^(p„^(pJ) < C<5,} = Pr{l - Sb{K, ~ l,n„ J^(pJ) < (6} 

> PrK^B {Ps.^iPs)) < Hcm = 1, 

1 > Pr{^p^(p„^(pJ) < C<5.} =Pr{5B(if.,n„^(pJ) < CS} 

> PrK.^B (p„^(p,J) < ln(C<5)} = 1 

and thus Pr{Fg^ (p^, '^(p^)) < CJ^, Gp^ (p^, (p^)) < C^s} = 1, which implies that {Dg = 1} is 
a sure event. So, the sampling scheme satisfies all the requirements described in Theorem [2l from 
which Theorem [28] immediately follows. 

1.15 Proof of Theorem [21] 

We need some preliminary results. 

Lemma 58 \^BiPe,Pe - £a) > Pe < p'' + £a \ = {z^ <Pi<P* + £a}- 
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Proof. By the definition of sample sizes, we have ~ ^^^p*^^ p*) and thus < Ug — I < 

HCS) ^ ln(C<5) 

Not.__„ - - 

monotonicaUy increasing with respect to z G {ea,z*) as asserted by Lemma [53l we can conclude 
from the intermediate value theorem that there exists a unique number £ (caiP* + ^a) such 
that ^Biza + £a) = ^'^nP ■ Finally, by virtue of the monotonicity of ^b{z, z — £a) with respect 
to z G {ea,z*'), the lemma is established. □ 

Lemma 59 [.^b (pe, ife) > Pi > + Sa} = {p* + ea<Pt< z+}. 

Proof. Note that ^-^{z* ■, i^^) = ^'b{z*,z* — £a) > ^^7^- By the definition of sample sizes, we 



_l£(C£) _ MC^) where z* = p* + £a- Since ^Biz*,z* — Sa) is negative, we have 

^b{z*,z* - Ea) > Noting that lim^^^e^ .^b{z, z-Sa) ^ -00 < i^^^ and that ^bIz, z — £„) is 



have 



ni 



ln(l/(l+£,)) 



and thus > m > , = = j. ,, , which 

ln(l/(l+£r)) .^B(l,l/(l+£r)) lim^^i ^B(2,z/(l+er)) ' 



implies liuiz^i ^b{z, < "^^ . Noting that -^b(-Zj 1^^) is monotonicaUy decreasing with 
respect to z G (z*, 1), we can conclude from the intermediate value theorem that there exists a 
unique number z^ G {z* , 1] such that ^b(2^, j^^) = '"^"^"^^ ■ Finally, by virtue of the monotonicity 
of ^Biz, ) with respect to z G (z*, 1], the lemma is established. 

□ 



Lemma 60 For £ = 1, - ■ ■ , s — 1, 



{0<Pe<P* - £a} for Hi < i^^^, 



l^B{Pi,Pi + £a) > Pi<P* -£a > ^ I {Z+ <Pi<p*- Ea} for < Tli < 



(p*-ea,P*) ' 



Proof. In the case of < j^^"/!*^^ ^ , it is obvious that ln(l-£a) > Since lim^^o -^b(^, ^+£0) = 

ln(l— Eq) < 0, we have limz_>.o ^b{z, z+e^) > Observing that ^b{z, z+Ea) is monotonicaUy 

increasing with respect to z G (0,p* — Ea), we have ^Biz,z + Sa) > ^^^^ for any z G [0,p* — Ea]- 
It follows that {^b(p^,P^ + £a) > < - = {0 < < - • 

In the case of ^^^^ < < „ /"^^^ we have < = , where 

z* = p* — Ea- Observing that ^Biz* , z* + Ea) is negative, we have ^b{z*,z* +ea) > ^"^^^^ . On 
the other hand, lini;,^o ^b{z, z + Sa) < as a consequence of > ^^("^'■f^^^) = nm.^o'^Jl^z+ea) ' 

Since ^b{z,z + Ea) is monotonicaUy increasing with respect to z G (0, z*) C (0, | — we can 
conclude from the intermediate value theorem that there exists a unique number zj" G [0,p* — Ea) 
such that .-#b(^^ , + £a) ~ ^'^nP ■ By virtue of the monotonicity of .^b(-2, z + Eq) with respect 
to z G (0, z*), we have {^B{Pe,Pi + £a) > Pe < P* - £a} = {z+ < < - £a} ■ 

In the case of > /"i*^*^^ we have > /"i'^'^^ = /"i'-'lL v Due to the fact that 

^ — ^b(P — £a,P ) ' — .^b(P — £a,P } .^b(2 ,2^ +£a) 

^b(-2*i-2* +ea) is negative, we have .Mb{z* ,z* ^- < ^'^nP ■ Since .^^Biz, z + Ea) is monotonicaUy 
increasing with respect to z G (0, z*) C (0, | — Ea), we have that ^b(2) z + Eq) < ^^^^^ for any 
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z E [0, z*]. This implies that ^^B{PeiPe + > ^^^^^i Pe < P* — ^aj = 0- This completes the proof 
of the lemma. 



□ 



Lemma 61 For i = 1, - ■ ■ ,s — 1, 

Pi, —-^ > Pe >p 



Pf \ \n{C5) ^ , 1 Up'' -ea<Pe<z-} forne< ^ 



ne ' "J |0 forni> 



^B(P*-ea,P*) ' 

-^B(p*-eo,P*) " 



Proof. In the case of < ^^^'pii^^ p^-, , we have Ji^^z* , = ^b(z*, 2;*+ea) = ^B(p*-ea,P*) > 
ii^^. Noting that lim^^i.e^ ^3(2:, j-fj;) = -00 < i^^^ and that ^3(2;, is monotonically 

decreasing with respect to z G (z*, 1 — e^), we can conclude from the intermediate value theorem 
that there exists a unique number z~ G (z*, 1 — e^) such that .^B(zr, 13^) = '"^""/^ ■ By virtue 
of the monotonicity of ./^bC-Z, y^-) with respect to z G (z*,l — £r), we have {^B(Pf,Y3^) > 
Pe>P* ~ £a} = {p* - ea < < z^}- 



In the case of > .^^('"iitl •>) ' ^^'^^ ■^bI^*,!^!;) - Noting that ^b(.z, j^) 



monotonically decreasing with respect to z G {z* , 1 — e^); we can conclude that ^b{z, j^^) < „^ 

Pe \ ^ l"(C'?) 



IS 



for any z G [z*, 1 — e^)- This implies that {^B(Pf, t3^) > , Pi> P* - £a\ = 0- The proof of 
the lemma is thus completed. 

□ 



We are now in position to prove Theorem [29l Clearly, it follows directly from the definition 
of Di that {£>£ = 0} = {.^b(P£,^(P£)) > U {.-#b(p«, ^(p^)) > ^^}- I* remains to show 

statements (I) and (II). 

With regard to statement (I), invoking the definition of ^(p^), we have 

|^B(P£,^(Pf)) > ^^^^1 = \-^B^i3i-eo)>^^^^^.Pt<P* 

= {z- <pi <p* + ea}yj{p* + Ea <Pe < z+} 
= Ua <Pi< = {ne z- <Ke< ne z+} 

where the second equality is due to Lemma [58] and Lemma [59l This establishes statement (I). 

The proof of statement (II) can be completed by applying Lemma [60] LemmaEUand observing 
that 

(.^b(p„'2^(P£)) > i^^j = i^BiPi,Pe + Sa)>^^^,Pe<p*-ea\ 

U (?'. A:) >^-' >"•--}■ 

This completes the proof of Theorem [29j 
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1.16 Proof of Theorem | 

We need some preliminary results, especially some properties of function ^{z,fi). 

Lemma 62 ^{z, z + e) is monotonically increasing with respect to z £ (0, ^ — and is mono- 
tonically decreasing with respect to z G — ^,1 — e). Similarly, ^{z,z — e) is monotoni- 
cally increasing with respect to z £ (e, ^ + and is monotonically decreasing with respect to 

Proof. The lemma can be established by checking the partial derivatives 
d^{z,z + e) fl 2e 



Oz [(. + f)(l-.-f)]n2 3 

d^{z, z — e) f 1 ^ 2e 



dz If.-m (^ _ 2£^l]2 V2 ^ 3 

□ 



Lemma 63 Let < e < ^. Then, ^{z,z — e) < ^{z,z + e) < —2e^ for z G [O, |] , and 
^{z,z + e) < ^{z, z - e) < -2e^ for z £ l] ■ 

Proof. By the definition of the function ^(., .), we have that ^{z,^) = -co for z G [0, 1] and 
/U ^ (0, 1). Hence, the lemma is trivially true for < z < e or 1 — e < z < 1. It remains to show 
the lemma for z G (e, 1 — e). This can be accomplished by noting that 

y.f N yy^ X 2e^{l-2z) 

J^{z^ z + e)- ^(z, . - .) = 3(, + |)(i_,_|) (,_|) + 

where the right-hand side is seen to be positive for z G (e, ^) and negative for z G (^, 1 — e). By 
Lemma [62l the maximums of ^{z, z + e) and ^(z, z — e) are shown to be — 2e^. This completes 
the proof of the lemma. 

□ 

Lemma 64 ^ (z, ■^) < {z, j^) <OforO<z<l-e<l. 
Proof. It can be verified that 



l + ej K'l-eJ 3(l + |)[l-3 + e(l-j)](l-j)[l-z-e(l-j)]' 
from which it can be seen that ^ {z, j^) < {z, < for z G (0, 1 — e). 

Lemma 65 Jl {^i — e , [i) < Jl {[i + e, [i) < —2e^ for 0<e</i<^<l-e. 



□ 
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Proof. The lemma follows from Lemma [63] and the fact that 

e^{2fi - 1) 



3(/.-|)(l-/. + |) + 

where the right-hand side is negative forO<e<;U<^<l — e. 



□ 



Lemma 66 ^{z, j^) is monotonically decreasing with respect to z £ (0, 1). Similarly, ^{z, j^) 
is monotonically decreasing with respect to z £ {0,1 — e). 



Proof. The lemma can be shown by verifying that 



l + e 



V'l + ey 2(l + §) [(l + e)(l_^) + 2|.]2 



< 



for z € (0, 1) and that 
d 



dz^V^T^e 



l-e 



< 



for z G (0, 1 - e) 



□ 



Lemma 67 For any fixed z £ (0,1), ^{z,fi) is monotonically increasing with respect to fi £ 
(0, z), and is monotonically decreasing with respect to fi £ {z, 1). Similarly, for any fixed £ (0, 1), 
^{z,^) is monotonically increasing with respect to z £ {0,fi), and is monotonically decreasing 
with respect to z £ {fi,l). 

Proof. The lemma can be shown by checking the following partial derivatives: 



d^{z, \i) 

d^{z, fi) 
dz 



{z-fi) [/u(l - z) + z(l - /i) + z(l - z)] 



2m I 



1 _ 2m 
^ 3 



in-z) /i(l 



2^ 



d + 



Z — fJ. 



(/x-z) (l-^)(f + + 



2/i _|_ £ 
3 3 



1 _ in 

^ 3 



2m I £ W 1 - 2££ _ £ 
3"'"3M^ 3 3 



□ 



Lemma 68 {Dg = 1} C {^b{Pi,'^ {Pi)) < 



UP„^iPe))<'-^} for£ = l,■ 



. s. 
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In 4- 2 

Proof. By the definition of Ug, we can show that Ug < -j^, which imphes that | + 2in{(S) — ^ 
for £=!,••• , s. It can be shown by tedious computation that 



ni 



7l£ 

for £ = 1, • • • , s. By (|106p . we have 

Pf, :j > , Pe> ^^r^) ea <Pe < —: — j > ■ 107) 

By the assumption that < < | and < Er < 1, we have |^ — < ^ — Hence, by 

virtue of p03p . we have 



Therefore, making use of p07|) and (jlOSp . we have 

} 



„*(P,))>^^} = U(p..r^]>'^.P,>'^-^.) (109) 



U <^ M(pi,,Pi + Eq) > , Pi< £a 

rif Er 



1_2 _ /I 6(l-£.)(3-£.)ln(C(5) 
2 3^° V 4 2 ln(C(5) 2(3 - ln(C'5) - Orife^ 



By (fT05]) . we have 

\ ln(C(5) ^ ^ £a ^ 1 [gq , , , 6(l+£,)(3 + £.)ln(C^) ) 

By the assumption that < < | and g^^g < Er < 1, we have + < | + Hence, by 
virtue of p04p . we have 



Therefore, making use of (jllip and (|112p . we have 

^(P„^(p,))>i^j = |^fp.-|^)>i^,P,>^ + J (113) 

ne ) [ \ 1 + Er J ni Er J 

,p,-e„)>^^, <-+£ 



i^L _ /I , ^^gg .c; ^ 6(l + £.)(3 + £.)ln(C<5) I 

2 + 3 " 1/4 21n(C<5) 2(3 + £,)2 ln(C<5) - Qn.eM ' ^^^^^ 
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It follows from ([TTO]) and ([TTi]) that 

{D, = 0} = f ^(p„^(p,)) > i^^l u > i^^l , (115) 

which implies that = 1} = {J^{p,,'W [p,)) < if (p,)) < ^} for £ = 1, • • • , s. 

So, 

{D, = l} = (^(p,,^(p,))<i!lM, ^(p^,^(p^))<l!^ 



for ^ = 1, • • • , s. This completes the proof of the lemma. 



□ 



Lemma 69 Dg = 1. 

Proof. For simplicity of notations, we denote = In view of (I109p . pi3p and (jllSp . we 
have that, in order to show Dg = 1, it suffices to show 

^ (ps, T^) > > - ea) = 0, (116) 

\ 1- SrJ Us J 

,,p^+ea)>^^^,Ps<P* -So] =iD, (117) 
■^(P-TT^l >^^,Ps>P' + ea]=^, (118) 
- Sa) > < + = 0- (119) 



By the definition of n^, we have n., > ^/"^"^^ .^ > -^t^t^tt^— tt . By the assumption on Sa 



and Er^ we have < < < ^ < 1 — ea- Hence, by Lemma [65| we have ^ {p* — ea,p*) < 
^ (p* + ea,P*) < and it foUows that 

ln(C<5) 



>^ip* + ea,pn>-^iP*-ea,P*)- (120) 



By (dzni), 



(121) 

Noting that ^ {p* — £a,P*) = ^ {p* — £a, irr^) and making use of the fact that ^{z, j^) is 
monotonically decreasing with respect to z G (0, 1 — e) as asserted by Lemma [661 we have 

TZi;) >'^iP*- ^a,p")} = {Ps <P"- ea}. (122) 
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Combining (11211) and (fT22D yields (fTT6D . By (fT20D . 



By the assumption on Sa and e,-, we have p* — £a < — Recalling the fact that ^{z, z + e) 
is monotonically increasing with respect to 2; G (0, ^ — ^) as asserted by Lemma \62\ we have 
that the event in the right-hand side of (jl23p is an impossible event and consequently, (I117P is 
established. By (fT20l) . 

(124) 

Noting that ^ {p* + ea,P*) = ^ (^P* + £0, ^^7^) and making use of the fact that ^{z, jj^) is 
monotonically decreasing with respect to 2; G (0, 1) as asserted by Lemma [Ml we have 

Ps,T^] >^{p''+ea,P*)]={Ps<P* + ^a}. (125) 



1 + ^^ , 

Combining (fM and (fM yields (fTTHD . By (fM . 

-^(Ps,Ps -£a) > Ps <P'' + ea\ C {^{p„p,-ea) > {p* + ea,P*), < P* + Sa} ■ (126) 

Z, 2: — e) 

is monotonically increasing with respect to z G (0, ^ + ^) as asserted by Lemma \62\ we have 
that the event in the right-hand side of (jl26|) is an impossible event and consequently, (jll9p is 
established. This completes the proof of the lemma. 

□ 



By the assumption on and , we have p* + £a < ^ + %^ • Recalling the fact that 



Now we are in a position to prove Theorem 1301 Note that ^b(-Z)P) = iiift>o E[e*^*] and 
that is a ULE of p for £ = !,••• , s. Moreover, {Dg = 1} is a sure event as a result of Lemma 
[69l So, the sampling scheme satisfies all the requirements described in Corollary [H from which 
Theorem [30] immediately follows. 

1.17 Proof of Theorem M\ 

We need some preliminary results. 

Lemma 70 lim^^-^o S^=i ''^i e~^'^ = for any c > 0. 

Proof. For simplicity of notations, define p* = |^ as before. By differentiation, it can be shown 
that xe~^'^ is monotonically increasing with respect to x G (0, i) and monotonically decreasing 

with respect to x G (^,oo). Since the smallest sample size ni > . is greater than ^ for 
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small enough e,. > 0, we have that X]^=i ^ "^'^ — ■^'^i ^ if er > is sufficiently small. Let 



p = inf£>o — 1. Observing that 



s < 1 



ln(l + p) 



In 



< 1 



In- 



ln(l + p) 



and ni > ^ , we have that 



^=1 



ln(l + p) 



■ exp 



C<5 



A{er) 



ln(l + er) ^\ ln(l + e^)/ c ln(l + p) 



S(£r), 



where ^(e,) = ^i^!^ exp (-j;^!^) and B(£,,) 



ln(l+e,) 



expf-TT^i;^). Notin: 



ln(l+e,) 



;ln 



that lima;_>oo ^ = and that . , '■^ . — oo as 0, we have lim£^_!.o ^(er) = 0. Now we show 



ln(l+e 

that limg^^^o B{er) = 0. Using Taylor's expansion formula ln(l + x) = x — ^ + ^ + o{x^), we 
have In = — ln(l + e^) = —£r + ^ + o{e^.) = —Sr + o{er) and 



+ 



2(p* + e„)(l-p*-ea) 3(p* + ea)2 3(l-p'^-ea)2 



2p*(l -p* 



+ wel + oiel), 



where vu = — ^ + tttt + ^tt-^-ttt. Hence, 

2p* 2(l-p*) Sp*-^ 3(l-p*)^ ' 



In 



1 



In- 



In. 



-Er- + T + o(^r) 



2p*(l-p*) 



ln[2p*(l -p^)] +ln — +ln 



1 . 1 - ^ + o(ea) 



ln[2(l - p*)] + In — + In , , -— 

^ ^ £a l-2p*{l-p*)wea + o{ea) 

ln[2(l - p*)] + In 1 + 2p*(l - p*)ti7e, - + o{ea) 



and 



In 



^+ea,P*) l+Sr 



IniTi;) ln[2(l-p*)]+ln^ 2p*(l - p*)n7ea - ^ + o(ea 



ln(l + e,) 



ln(l + Er 



+ 



ln[2(l-p*)/p1+ln^ _ ^2,^ 1 



ln(l + 



+ 2p*\l-p*)uj-- + oil). (127) 



Making use of p27p and observing that 



2p*2(i_p^)^__ + o(l) 



exp 



ln(l + e^) 



0(1), 
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ln[2(l 

\n{l + er 



exp 



ln(l + Er 



cln^ 



ln[2il -p*)/p*] TKiTT^) 



C<5 



exp 



we have 



B{er) = o(l) + 
= o{l) + 



ln(l + £r) 

er + o(£r) 



exp 



exp 



ln(l + Er 



0(1) 



■ exp 



oil), 



cln-i 



C<5 



l + y + «(e,.) 



Inf / 1 / 1 \-f[i+°Wl 



e.r + o{er) \C5 

B*{Sr) f 1 



l + o(l) \C6 



where B*{er) 
we have 



In-L 



Making a change of variable x = — and using L' Hospital's rule, 



lim B*{er) = lim ^\^cx 



lim 



1 + Inx 



(\ / \ cx lim ^ 2 y 



Therefore, < limsup,^,^o J^Ui 6""''= < ^ lime,_,o A{er) + y^^jj^ x (^^j x lime.^o B*(£r) = 0, 
which implies that lim^^^Q ^£=1 e""'^'^ = 0. This completes the proof of the lemma. 

□ 



Lemma 71 // £a is sufficiently small, then the following statements hold true. 



(I): For 1 < i < s, there exists a unique number zi G [0,p* — Sa) such that ni = 



for ui > 



ln{l--£a) ■ 



Ze, Zl+Sa) 



(II) : For 1 < I < s, there exists a unique number yi € (p* + 1] such that = 

(III) : Zi is monotonically increasing with respect to i; yi is monotonically decreasing with 
respect to i. 

(IV) : lim^^^Q Z£ = \/i 4p ^(i p lm\^^Qyi = — / i ^ \ > where the limits are 

taken under the constraint that |^ and s — i are fixed with respect to £a- 

(V) : Let is = s — jp. For p £ (p*, 1) such that Cj^ = r{p), 

lim ~^ = '^ P~P* 

£r-s>0 ErP 31 — p* 

For p G (0,p*) such that Cj^ = r{p), 

lim ~P = 2p(l-p)(l-2p'^) _ 2 
ea™0 Ea 3p*{l-p*){l-2p) 3' 
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(VI): 



{De = 0} = 



I {zi <pi< ye} for ne > n^p^z^; 
[{0<p,<y,} forne<,^. 



Proof of Statement (I): By the definition of sample sizes, we have 



^ (1 + ClK / (1 + Cl) 
'^i < 7. < 7. 



ln(C5) 



+ 1 



> ./#b(0, Eq) and 



(128) 



for sufficiently small > 0. As a consequence of (jl28p . we have 



ln(C<5) 



< 



'*+ea, P*) i^r^ 

\1 + Cl Hi 



^B(p* + ea, P") 



rii ' - yi + Oi Hi J ^b(p* - Ea, P*) \1 + Cl 

provided that > is sufficiently small. Noting that 



*-£a, P*)-'- 



ijP* +£a, P*) 

ne 



^BiP* +£a, P*) 
lim r- 

ea^O JIbKP* - £a, P*) 



, ^B{P* + £a,P*) „ 

1, Irni = 0, 



we have that 



< ^sip* — £a, P*) for small enough > 0. In view of the established fact that 



./#b(0, Ea) < ^^nP < (p* — £a, P*) and the fact that ^-b{z, z + Sa) is monotonically increasing 
with respect to z G {0,p* — Ea) as asserted by Lemma[T6l invoking the intermediate value theorem, 
we have that there exists a unique number zg G [0,p* — Sa) such that .y^BizeTZg + Ea) = ^-^^p-^ 
which implies Statement (I). 



Proof of Statement (II): By the definition of sample sizes, we have 



ln(C5) 



•^B(l,Tq^ 



O + CiK (1 + Ci) 
< ni < Hi < < 



ln(C<5) 



^b{P* + Ea, P* 



+ 1 



(129) 



and consequently, '"^'^'^^ > 



HCS) 



ne 



< ^b{p* + £a, P*) 



1 



1 + Cl Jie 



1 + 



for sufficiently small Ea > 0. Noting that lim^ 



?B P 



^b{p* +£a, P*) 



ne 



0, we have 



< ^b{p* 



^^b{p* 



for small enough Ea > 0. In view of the established fact that ^b{^, < 



< 



p +ea 



and the fact that ^^{z, 



is monotonically decreasing with respect to 



z G (0, 1) as asserted by Lemma HOl invoking the intermediate value theorem, we have that there 



exists a unique number yi G (p* + e^, 1] such that ^b^Mi 
(II). 



yt 

l+er 



ln(C'?) 

Til ' 



which implies Statement 



Proof of Statement (III): Since ni is monotonically increasing with respect to ^ if > 
is sufficiently small, we have that ^^(^ze^ze + Ea) is monotonically increasing with respect to I 
for small enough Ea > 0. Recalling that ^Biz,z + Ea) is monotonically increasing with respect 
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to z £ {0,p* — £a), we have that Zi is monotonically increasing with respect to I. Similarly, 
-^BiVii 1+77) is monotonically increasing with respect to (. for sufficiently small Sa > 0. Recalling 
that -#b(-2)]:^) is monotonically decreasing with respect to z G (0,1), we have that yi is 
monotonically decreasing with respect to i. This establishes Statement (III). 

Proof of Statement (IV): We first consider lim^^^Q ze. For simplicity of notations, define 
bf = P £ ^ < s such that np > t-tt^-^- Then, it can be checked that ^ll = 

•^2 — In(l-eQ) ' p*{l-p*) 

Cs-i and, by the definition of sample sizes, we have 

hi,{l-hi,) J^B^Zi^Zi + Ea) 1 Cs_^ln(C5) 

-, r , r = X r = 1 + o(l) (130) 

p'^(l -p^)^B(p* + ea,P*) Ui ^Bip* + ea,P*) 

for i < s such that ni > '"^^"^^ 



ln(i-£a) • 

We claim that Z£ > 6 for 6 € (0,6^) provided that > is sufficiently small. Such a claim 
can be shown by a contradiction method as follows. Suppose this claim is not true, then there is 
a set, denoted by S^^, of infinitely many values of Ea such that zi < 9 for any Sa € S^^- For small 
enough G S^^, it is true that Z£ < 6 < bi < ^ — Sa- By (jl33p and the fact that z + e) is 

monotonically increasing with respect to z € (0, ^ — e) as asserted by Lemma [T6l we have 

bi{l - bj) ^Bjze, Zj + Eg) _ ^ ^ bi{l-be) ^3(0,0 + £a) _ bijl - bj) 

p*{l-p*)^B{p* + ea,P*) ~ ' - p*{1-p*)^b{p* + £a,P*) ~ 0{l-e) ' 

for small enough G Ss^, which implies ^g^^Zffj ^ 1; contradicting to the fact that ^g^^Zffj > 1- 
This proves the claim. Now we restrict Sa to be small enough so that 6 < z^ < p* . Making use of 
(|133p and applying Lemma [15] based on the condition that 2£ G {0,p*) C (0, 1), we have 

6,(1 -M el/[2z,{z,-l)]+o{el) 
p*{l-p*) el/[2p^{p*-l)]+o{el) ^ ^ 

which implies = 1 + o(l) and thus lime^_j.o = b(^. 

We now consider lim£^_s.o Hi- For simplicity of notations, define ai = — f \ ^r 1 < I < 

i+(F^-ijC''-f 

s. Then, it can be checked that ^j^^^^ "^""'^ = Cg^i and, by the definition of sample sizes, 



l-p* an ^B{p* + ea,P*) ni ^B{P* + £a,P*) 

We claim that yi < 9 for 9 € (a,, 1) if > is small enough. To prove this claim, we 
use a contradiction method. Suppose this claim is not true, then there is a set, denoted by S^^, 
of infinitely many values of £r such that yi > 9 for any Sr G Se^- By ()132p and the fact that 
^b(z, Y^) is monotonically decreasing with respect to z G (0,1) as asserted by Lemma HOl we 
have 

p^ I -a, ^Biyi,T^) P" l-ai ^Bi0,T^) _ 0{1 - ae) ^ ^^^^ 



1-p* Qi ^B{p* + £a,P*) 1-p* 0£ ^b{p* + ea.,P*) ae{l - 9) 
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for small enough e,. G 5^^, which implies 



< 1, contradicting to the fact that 



> 1. 



This proves the claim. Now we restrict £r to be small enough so that p* < ye < 0. By (jl32p and 
applying Lemma [15] based on the condition that € {p*,0) C (0, 1), we have 

P* 1 - «^ X - 1)] + o(£r) ^ ^ ^ ^^^^ 



1-p* a, e2/[2p.(p._i)] + o(e2) 
which imphes aT(i'-ye) = "(-^^ ^^^^ hm^^^o Ue = ae- 
Proof of Statement (V): 

We shall first consider p G [p*, 1). For small enough > 0, there exists Z£^ G (p*, 1) such that 



ln(C<5) 



ze,,ZiJ{l + er)) 



C. 



ln(C5) 



P 



1-p 



1 — p* p 



,P*/{l+er)) 



rBip^p'/il+Er)) 

For 9 G (p, 1), we claim that < ^ if is sufficiently small. Suppose, to get a contradiction, 
that this claim is not true. Then, there exists a set, denoted by Sg^, of infinitely many values of 
Er such that zg^ > 9 for any value of Er in S^^ ■ Noting that 

nrrr n" \ — n ^^r, \ 7« r« / 1 1 -4- ;r„ n 

(132) 



i-p* p . 



,(p*,pV(l+er)) 



p* l-pJi^B{zi,,Z(,J{l+Er)) 



we have 

p* 1 - p^B{ze,,ZiJ{l + Er)) 



1-p* p ^B(p^pV(l + er)) 



1 + 0{Er), 



1 — p* p 



,pV{l + er)) 



. . . 1-p ^B{0,0/{l + er)) _ 9{l-p) 

^"^ '■^-1-p* P ^B(p^pV(l + er)) "p(i-^)^''^ ^ 



for any value of Er in Sg^, which contradicts to the fact that > 1. This proves the claim. 

Now we restrict Er to be small enough so that p* < < 9. Since is bounded with respect to 
E, by (|132p and Lemma [151 we have 

-E^rZijm-za+E^rZiA'^-Zi,)/m-Ze^f]+o{4) _^ ^ ^^^^^^^ 



1 — p* p 



I.e., 



I.e., 



I.e., 



-e2p*/[2(l - p*)] + e3p^(2 - p*)/[3(l - p*)2] + o(e3) 

(I-P) 2£^2f^(l-p)(2-2:<;J _|_ ^^^^^ 



1 + o(er-), 



l-2e,(2-p*)/[3(l-p*)] + o(e,) 
-p 26^24(1 -p) (2 - z^) 26^(2 -p 



p(l-Z4) 3p(l-Z4)2 3(1 -p 

^-p 2e,Z4(l-p)(2-Z4) 2e,(2 - p'^)(l - 



+ OiEr 



p 3p{l-ZiJ 3(1 -p* 

which implies that lim£^_^o ze^ = p and consequently, 



+ o{Er), 



lim 



p 2(2 -p) 2(2-p*)(l-p) 2p-p* 



er^O ErP 



3(1 -p* 



3 1 - p* 
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Next, we shall consider p G (0,p*). For small enough Sa > 0, there exists G (0,p*) such 



that 



ln(C<5) 



+ ^a) 



C._4 ln(C<5) 



ln(C5) 



p*{l-p*) ^b{P*,P* + £a) 



For ^ G (0,p), we claim that > 9 i( Sa is sufficiently small. Suppose, to get a contradiction, 
that this claim is not true. Then, there exists a set, denoted by Sg^, of infinitely many values of 
£a such that Z£, < 9 for any value of in Se^ ■ Noting that 



P(l-P) ln(C.5) n \ yy r , ^ 
v*(l-v*) Mv^p* + ea) 



p*{l — p*) 



(133) 



we have 

p{l -p) 



P*{1-P*)J^b{p\P* + ea) 9{l-9) 



for any value of Sa in , which contradicts to the fact that > 1 . This proves the claim. 

Now we restrict to be small enough so that 9 < z^^ < p* . Since zi^ is bounded with respect to 
e, by (jl33p and Lemma [T5l we have 

p{l-p) -gg/[2z4(l - zg + 4(1 - 2ze^)/[3zl{l - z,^f] + o{eI) , , 



I.e., 



I.e., 



I.e., 



I.e. 



p-{l - P-) -el/[2p-{l - p*)] + £3(1 _ 2p-)/[3(p*)2(l - p-)2] + o(£3) 
P(l-P) _ 2eapil-p)(l-2zeJ ^ ^^^^^ 



2«, 



1 - 2£,(1 - 2p*)/[3p'^(l - p*)] + 0(£a) 



1 + 0{£a), 



p{l-p) 2eap{l-p){l-2ze^)_ 2ea{l-2p*) 



Z4(l-Z4) 3^(1-^4)2 ^ 3p*(l-p-) 

(z^ - p){l - - p) 2£,(1 - 2p*) 2eaP{l -p){l- 2ze, 



^4(1-^4) 3p'^(l-p'^) 3z|Jl-Z4)2 

Z4 - p 2Z4 (1 - Z4 ) (1 - 2p*) 2p(l - p) (1 - 2z4 



£a 3p*(l-p*)(l-Z4 -p) 3Z4(1-Z4)(1 

which implies that lim£^_>.o Z£^ = p and consequently, 

ze^-p 2p{l-p){l-2p*) 2 



lim 



3p*(l -p*)(l - 2p) 3 



G 



P) 



,0 . 



+ o(£a), 
+ 0(1), 
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Proof of Statement (VI): By the definition of the samphng scheme, 
{De = 0} = (max{^B(P£,^(Pf)), ^B(Pf, '^(p^))} > \Pe - P*\ < Sa 

|j|niax{.^B(p„=S^(p£)), .^b(p„^(p,))} > i^^, <p*-ea| 

= |max|.^B(P/?,P^ - £a), J^y^ | > \Pi~P*\ - ^-^j 

|j|.^B(Pf,P^ +ea) > ^^^1^' Pi <P* -~£a^[j^-^B (pi^ J^^^ > ~~ ' P^ > P* + ^^a^- 

We claim that if > is sufficiently small, then it is true that 

max|.^B(Pf,Pf-£a), (pe^Y^^^^ > ^^7^' \Pe^P*\ < e^j = {\Pe~P*\ < £a} , (134) 

-^B(Pf,P£ +£0) > i^^^^, Pi <p* - Sal = {^i! <Pl<P* - Sa) for . ^"^'"'^^ , < < n^, (135) 

ni J In(l-eQ) 

^ N ln(C'5) ^ . 1 r ^ . . ln(C'5) , , 

^B{Pl,Pt + Ea) > ^ Pi < P ~ £a> = {0 < Pi, < p - Ea} for Til < m < ■ ,/ (136 

?i£ J ln(l-£a) 

( ) > i^^^, > + £a I = {p* + ea<Pi< Vl} ■ (137) 



To show ()134p . note that 



^ (1 + Ci)n, (1 + Ci) 



(138) 



which implies that 



ln(C.5) ^ ^B{p* + ea,p*) [ 2 \ , ^ ^ . ^Bip* + ea,P* 



ne ^b{p* - £:a, P* - £a - £a) + Ci J ni 
if Sa > is sufficiently small. Noting that 

2 

j.^ ^B{p*+£a,P*) ^ ^.^ 2pH;*-1) + 0(4) 



and lim^ ^ q 

<^B(p*-ea, P*-ea-ea) (139) 

for small enough Ea > 0. Again by (llSSp . we have 

ln(C(5) ^B{p* + ea,P*) ( 2 \ / . P^ + ^aA ^B(p* + ea,P'' 



n£ + Vl + Ciy °V 1-e. y ne 
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if £a > is sufficiently small. Noting that 

2 



and lim, -^^(p^+^a.p*) = q, we have 



^^(^')<^Bfp^ + ..,^) (140) 



for small enough Sa > 0. It can be seen from Lemmas [TBI and [55] that, for z G [p* — Sa, P* + Ea], 
^b(-2,^; — £a) is monotonically increasing with respect to z and -^b(-2, jrf^) is monotonically 
decreasing with respect to z. By (|139D and (I140P . we have < ^b{z,z - e^) and ^^K^i < 

^b(-2, rr^) foi' s-iiy 2 G [p* — ea,P* + Ea] if Ea > is small enough. This proves p34p . 

To show (|135p . let co £ {./#b (p^iP^ + ^^a) > ^^^^ , Pe < p* - Ea} and pe = Pi{i^)- Then, 
•^b(p^, P^+^a) > ^^^^^ andp^ < p*—ea- Since G [0,p*— e^) and .^#3 (-2, z + Eq) is monotonically 
increasing with respect to z G (0,p* — Ea), it must be true that pi > Z£. Otherwise if pi < 
Zi, then (pi,P£ + Ea) < -^B (zi, Zi + Ea) = ^^^^ ; leading to a contradiction. This proves 

{^B {Pl3l> + Ea) > ^^7^, < - Ea} {z£ < Pi < p* ~ Ea}- NoW let LJ £ {z^ < < p* - Ea} and 

Pi = Piiu})- Then, zi < pi < p* — Ea- Noting that (z, z + £„) is monotonically increasing with 
respect to z G (0,p* — Ea), we have that {pi,Pe + £a) > izi,Zi + £a) = ^'^nP , which implies 



B {Pi,Pe + Ea) > ^-^,Pe <P* - Ea} 3 {zi < P(, < p" ~ Ea}- Thls cstabllshes (USS])- 



71 £ 



Note that, for any z G (0,p* — Ea), we have .J^b{z, z + Ea) > ^B(0,ea) = ln(l — Ea) > ^^n^ , 
which implies ([136]). 

To show (HSID, let oj e l^BiPi, ifc) > Pe > P*+Ea} andpe = M^)- Then, ^Bipe, jf^) > 

^'^l^P and Pi > p* + Ea- Since ye G (p* + ea, 1] and .-#B(z,Yq^) is monotonically decreasing 
with respect to z G (p* + Ea,l), it must be true that pi < ye- Otherwise if pe > Ui, then 
•^b(w,Y^p) < -^bCj/^,]^^) = '"^'^'^"^ , leading to a contradiction. This proves {^B{Pe, j^^^) > 
i^^, pg> p* + Sa} c {p* + < < ye}. Now let G {p* + ea < < and pe = Pe{uj). Then, 
p* + Ea < Pe < ye- Noting that .^^Biz, jj^) is monotonically decreasing with respect to z G (0, 1), 
we have that .^b(p,,^) > ^^(y,,^) = Isgll, which implies l^BiPe,j^) > Pe > 

P* + Ea} 3 {p* + Ea < Pe < Ve} - This establishes (|137p . 



Lemma 72 Let ig = s — jp. Then, under the constraint that limits are taken with |^ fixed, 

4-1 s 
lim V ne PT{De = 1} = 0, lim V ne PT{De = 0} = (141) 

for p G (0, 1). Moreover, lim£^_^o'^4 Prj-D^ = 0} = if Cj^ > r{p). 
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Proof. For simplicity of notations, let ae = lim^^^QUi and = lims^^^o Z£. The proof consists 
of three main steps as follows. 

First, we shall show that (jl4ip holds for p G {0,p*]. By the definition of ie, we have r{p) > 
Cs-e,+i- Making use of the first four statements of Lemma [TTl we have that zi < ^^^^^-^ < p for 
alH < - 1 with > and that yi > P-±^ > for 1 < ^ < s if is sufficiently small. 

Therefore, by the last statement of Lemma ED and using Chernoff bound, we have that 

Pr{De = 1} = Pr{p, < + Pr{p, > ye} < Pr < P±^iZ± | + > ^1^^ 

< exp —2ne + cxp — 2n^ p 



for all ^ < £e — 1 with > ■) and that 

Pr{D, = 1} = Pr{p, > y,} + Pr{p, = 0} < Pr |p, > | + Pr{p, = 0} 

< exp ^-2ni ^2^'^ ~ ^) ^ + *^^P(~^'^^?'^) 

for all H. with < i^^^^^ ^ if > is small enough. As a consequence of the definition of Ig, we 
have that 64-1 is smaller than p and is independent of > 0. Hence, we can apply Lemma [70] 

to conclude that lim£^_>o J2i=i^ 'T-f Pr{I?f = 1} = 0. 

Similarly, it can be seen from the definition of that r{p) < Cg^i^-i. Making use of the first 
four statements of Lemma [7T1 we have that Zi > > p for £g + 1 < £ < s if is sufficiently 

small. By the last statement of Lemma [7T] and using Chernoff bound, we have 

Pr{D, = 0} = Pt{z( <pe< yt} < Pr{p, > z,} < Pr |p, > ^±^| < exp |^-2n, (^^^-^^ ^ 

for ig + 1 < i < s if Ea > is small enough. By virtue of the definition of ig, we have that 64+1 
is greater than p and is independent of Ea > 0. In view of this and the fact that PrjD^ = 0} = 0, 
we can use Lemma [70] to arrive at linie_^_i.o J2i=e^+i '^^ Pi'{-Df = 0} = 0. This proves that (|14ip holds 
for p G (0,p*]. 

Second, we shall show that (|14ip holds for p G {p*, 1). As a direct consequence of the definition 
of is, we have r{p) > Cg-i^+i- Making use of the first four statements of Lemma [TTl we have that 
Vi > > p for all i < is — I and Zs-i < ^ "^2°^^ < P* if £a is sufficiently small. By the last 

statement of Lemma [7T] and using Chernoff bound, we have 

FT{De = 1} < Pr{p^ > ye} + Pr{p^ < ^s-i} < Pr > P±^h^ \ + Pr < Pl±pzl 



< exp I -2ne ( 1 I + exp I -2n£ Ip 1 
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for all £ < if, — 1 provided that > is small enough. As a result of the definition of ie, we have 
that o^-i is greater than p and is independent of Ea > 0. Hence, it follows from Lemma PfOl that 

lime^^o Y^e^Ji ni Vr{Di = 1} = 0. 

In a similar manner, by the definition of £e, we have r(p) < Cs-e^-i- Making use of the first 
four statements of Lemma \T\\ we have that < < p for + 1 < £ < s if is sufficiently 

small. By the last statement of Lemma [7T] and using Chernoff bound, we have 

Pr{£), = 0} = Pr{z, <p,< < Pr{p, < y,} < Pr (p, < P±^!i±il < cxp ( -2n, " 



for £e + 1 < £ < s if > is small enough. Clearly, Pr{D<j = 0} = 0. As a consequence of 
the definition of i^, we have that a^^+i is smaller than p and is independent of Ea > 0. Hence, it 
follows from Lemma [70] that limEa^oJ2i=e Pr{Df = 0} = 0. This proves that (jl4ip holds for 
p€{p*,l)- 

Third, we shall show lim£^_j.o n^^ Pr{Z)£^ = 0} = for p G (0, 1) such that Cj^ > r{p). 

For p G (0,p*) such that Cj^ > r{p), we have r{p) < Cs-e, because of the definition of i^. 
Making use of the first four statements of Lemma [TTl we have that z^^ > ^'^2'^' > p if ea > is 
small enough. By the last statement of Lemma EU and using Chernoff bound, we have 

Pr{D,^ = 0} = Pr{z,^ < p» < ye J < Pr{p,^ > ZiJ < Pr U,^ > ^^^} < cxp (-271, " 



Since 64 is greater than p and is independent of > due to the definition of le, it follows that 
lim,^^on4 Pr{r>4 =0} = 0. 

For p G (p*, 1) such that Cj^ > r{p), we have r{p) < Cs-i, as a result of the definition of Is- 
Making use of the first four statements of Lemma [71} we have that yi^ < ^^^^^ < p if > is 
sufficiently small. By the last statement of Lemma [7T] and using Chernoff bound, we have 

Pr{D4 = 0} = Pr{z,^ < p,^ < J < Pr{p,^ < < Pr < ^^^} < cxp (^2n,^ 

Since ag^ is smaller than p and is independent of > as a consequence of the definition of ^e, it 
follows that lim£^_j.o n^^ Pr{Z)^^ = 0} = 0. This proves lime^_^o "^ie Pi'i-^^ = 0} = for p G (0, 1) 
such that > r{p). The proof of the lemma is thus completed. 

□ 



The proof of Theorem [3T] can be accomplished by employing Lemma[72]and a similar argument 
as the proof of Theorem [T5j 

1.18 Proof of Theorem [32] 

As a result of the definitions of Kp and we have that Kp > 1 if and only if r{p) is not an 
integer. To prove Theorem 1321 we need some preliminary results. 



Lemma 73 linie^^o A/-„,(pX.£r) = ^P' 1™-'«^o = lim^,._,o = 
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Proof. First, we shall consider p G (0,p*]. By the definition of sample sizes, we have 

li^ ^ = 1 (142) 

for 1 < £ < s. It follows that 

linl _ jjj^ ./#B(p,P + £a) ^ ln(C'^) _ j.^ Cs-4-^B(p,P + £a) 

ea->0 A/m(p,ea,er) ~ ^a^O ln{CS) ^BiP* + Sa^P*) ~ 1^0.^0 ^B{p*+ea,P*) 

^. C,-i. {el/[2p{p-l)]+o{el)) 
e™o sl/[2p*{p*-l)] + oiel) 



P*i^-P*)^ _P*{1-P*) 

and 



Ea-i-o Vp(1~-P) y p{l - p) ^Bip* + ea,p*) 



Next, we shall consider p £ (p* , 1]. By virtue of (jl42p . we have 

Cs^,^ (£,V[2(p-l)]+o(g,^)) 
e™o el/[2p^{p*-l)]+oiel) 



and 



hme.,/!^ = lime. ' ^ C^^^MC^) 



Sr^O yl—p er-s>0 y 1 — p ^b{P* + £a,P 



sr^o'Vl-p el/[2p-{p^-l)] + oiel) \p*{l-p)''' 



□ 



Lemma 74 Let U and V he independent Gaussian random variables with zero means and unit 
variances. Then, for p G (0, 1) such that Cj^ = r[p) and jp > 1, 

lim Pr{Z = 4} = 1 - lim Pr{Z = 4 + 1} = 1 - $ (ud) , 
£^0 e-s>0 

lim [Pr{|p,^ -p\>ep,l = 4} + Pr{|p,^+i - p\ > e^, I = + 1}] 

= Pr{C/>4 + Pr{|C/ + ^y| >dy^lTp^, U <i^d}, 

where Ep = ma-K{ea,£rP}- 
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Proof. First, we shall consider p £ [p*, 1). Since Kp = 1, by Statement (V) of Lemma [TTl we 
have 

lim — ^ - = lim i / F^^^ lim — — — = d lim — — - = vd. 



Er-^O y^p{l — p) /n^ Sr-^O ^ 1— p er->0 E^P £r->0 E^p 

Note that 
Therefore, 



Pr{|p4 -p| > ep, Z = 4} + Pr{|P4+i - p| > Sp, Z = 4 + 1} 
Pr{|[/| > 6, U > vd] + ¥i {\U + ^V\ > dyTTp^, U < i^d} 



= Fr{U>d} + Fi{\U + ^V\>d^/TTp^,U<ud} 
for p £ (p* ,1) such that Cj^ = r{p). 

Next, we shall consider p G (0,p*). Since Kp = 1, by Statement (V) of Lemma [Ti\ we have 



lim — = lim Ea\ —, — — — r lim — — = d lim — — = —ud. 

ea">0 i^a^O yp[l-p)ea^O Ea Sa^O Ea 



Note that 



r. ri-- 1^ ^ 1 D J \Pe,-P\ ^ / T^e, Pe, - P , ze, - P 



yp(l - Vp(I-p)' v/p(l-p)/"«e Vp(l-P)/'^fe J ' 

Therefore, Pi'lD^ = 1} Pr{C/ > z^d} and 

Pr{|p4 -p| > Ep, Z = 4} + Pr{|P4+i -p| > ep, / = 4 + 1} 



-)■ Pt{\U\ >d,U< -ud} + Pr{\U + > dsj\ + pp, C/ > -z^d} 

= Pr{C/>4 + Pr{|C/ + ^y| >dyrT7^, t/<z^(i} 

for p G (0,p*) such that Cj^ = r(p). 



□ 



Now, we shall first show that Statement (I) holds for p G (0,p*] such that Cj^ = r{p). For this 
purpose, we need to show that 

1 < lim sup .r ,^ ^ 7 < 1 + Pp for any S i lim p = p \ . (143) 

To show limsupg^_j.Q ^"^^^^ ^ ^ > 1, note that Cg-e^+i < r{p) = Cs~e^ < Cg^e^-i as a direct 
consequence of the definitions of 4 and jp. By the first four statements of Lemma UT\ we have 
lim£^_>.o < p for all ^ < 4 ~ 1 with > j ^"^ff.^ . . Noting that lim£^_!.o p(ci;) = p, we have 



l(l-e<.) ' 



p(c<j) > Zi for all ^ < 4 ~ 1 with 7i£ > j s and it follows from the definition of the sampling 
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scheme that n(a;) > ng^ if Sa > is small enough. By Lemma [73] and noting that /vp = 1 if 

Cjp = r{p), we have limsup.^^o A/-„("^,"ie.) > lime„^o ^f^^pX,e^) = «p = 1- 

To show linisup^_^_j.g "^^^ ^ ^ < l + Pp, we shall consider three cases: (i) is = s; (ii) is = s — 1; 
(iii) is < s — 1. In the case of is = s, it must be true that n(t<;) < rig = n^^. Hence, 
limsup^^_^o V„("';ei£,0 - l™ea->o A/.„(p,e„,£r) = = 1 = 1 + Pp- t^c casc of 4 = - 1, it must 
be true that n{uj) < Us = n^+i. Hence, limsup^.^^o A/'„("'eie,) ^ limg^^o j^JlpJ^,,^) = 1 + Pp- 
In the case of 4 < -5 — 1, it follows from Lemma 1711 that lim^^-^oz^+i > p, which implies 
that Zi^^i > p, p{oj) < Z£^^i, and thus n(a;) < n^+i for small enough Sa > 0. Therefore, 

l™«^Pe.->o mJ:L) < l"^e„^o j,J;::,e.) - ^ X lim,„^o = ^ + Pv This es- 

tablishes ()143p and it follows that {1 < lim supg^_j,Q ^ ^ ^ < 1 + Pp} 5 {lim£^_^oP = p}- 
According to the strong law of large numbers, we have 1 > Pr{l < limsup^^^Q ^ ^ < 

1 + Pp} > Pr {lim^^^^oP = p} = 1- This proves that Statement (I) holds for p £ (0,p*] such that 
C,,=r{p). 

Next, we shall show that Statement (I) holds for p G (0,p*] such that C^^ > r{p). Note that 
Cs-e,+i < r{p) < Cs-e, as a direct consequence of the definitions of is and jp. By the first four 
statements of Lemma 1711 we have lim^^^o-z^-i < P and thus zi < p for all i < is — 1 with 
"n-e > \n{i-^e ) provided that > is sufficiently small. Therefore, for any uj G {lime^_>oP = p}, 
we have zi < p{uj) < yi for all i < is — I with Ui > i^'"/!^^ ) and consequently, n(a;) > ni^ 
provided that > is sufficiently small. On the other hand, we can show that n(a;) < ni^ 
if > is small enough by investigating two cases. In the case of is = s, it is trivially true 
that n(a;) < n^^. In the case of is < s, we have p < linis^-^o zi^ and thus p < z^^ provided 
that > is sufficiently small. Therefore, for any w G {lim£^_>oP = p}) we have p{bS) < zi^ 
and consequently, n(tj) < ni^ provided that > is sufficiently small. So, we have established 
that n(a;) = ng^ if > is sufficiently small. Applying Lemma [73l we have Xvsn^^^Q ^ ^^^^ ^ ^ = 
lim£^_j.o ^^["^ ^ ^ = Kp, which implies that {liinj^_j.o ^-^ = Kp} D {linig^^o P = v\- It follows from 
the strong law of large numbers that 1 > Prjlinie^^o j~j (p" ^ ) = > Pr{linie„^oP = p} = 1 
and thus Pr{linie^_i.o ^ ^ ^ = Kp} = 1. Since 1 < < 1 + pp, it is of course true that 
Pr{l < limsupg^_j.o ^ ^ < 1 + Pp} = 1. This proves that Statement (I) holds true for 
p G (0,p*] such that Cj^ > r{p). Thus, we have shown that Statement (I) holds true for p G (0,p*]. 

In a similar manner, we can show that Statement (I) is true for p G {p*, 1). This concludes 
the proof for Statement (I) of the theorem. 

To show Statements (II) and (HI), we can employ Lemmas 1721 [73l and mimic the corresponding 
arguments for Theorem [16] by identifying and £rP as e for the cases of p < p* and p > p* 
respectively in the course of proof. Specially, in order to prove Statement (HI), we need to make 
use of the following observation: 

^ ,^ ,^ ,^ . |Pr{|p-p| > e^} forpG(0,p*], 

Fl{\p - p\ > Ea, \p - p\ > £rP} = { 

I Pr{|p — p\ > Erp} for p G (p*, 1) 
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p{l — p)J' t \ ^ — P j 

where, according to the central hmit theorem, 11^= . converges in distribution to a 

i/p(l-p)/nf 

Gaussian random variable of zero mean and unit variance as Sa — > 0. 



1.19 Proof of Theorem I 

We need some preliminary results. 

Lemma 75 Let Xn = — where Xi, ••• , X„ are i.i.d. random variables such that < 
Xi < 1 and E[Xi] = fi e (0,1) for i = 1, ■■■ ,n. Then, Pr {X„ > n, (Xn, A*) < ^} < a /or 
any a > 0. 

Proof. For simplicity of notations, let F-^^(z) = Pr{X„ < z}. By Lemma [H we have that 
{Xn > /"} = {^n > ^Y^iXn) < exp (n^B {Xn,IJ'))}- Therefore, 



— ,^ /— N In a 

^ ' n 



C {F-^JX„)<a} 
and thus Lemma [751 follows from Lemma [2j 



□ 



Lemma 76 Lei X„ = — -, where Xi, ••• , Xn are i.i.d. random variables such that < 
Xi < 1 and E[Xi] = ^ g (0, 1) /or i = 1, • • • , n. T/ien, Pr {X„ < ^u, ^b (^n, /«) < ^} < a /or 
any a > 0. 

Proof. For simplicity of notations, let Gy^{z) = Pr {X„ > z}. By Lemma [U we have that 
{Xn < /"} = {Xn < Gx^{Xn) < cxp (n^B (Xn,A^))}- Therefore, 

C {Gj^^(Xn)<a} 
and thus Lemma [76l follows from Lemma [2j 

□ 

Now we are in a position to show Theorem [33l By a similar method as that of Lemma [8l 
we can show that {^b (| ~ |^ ~ A^s| j i ~ |i ~ Ms| + ^ "TT^i ^ ^^'^^ event. By a similar 
method as that of LemmaO we can show that {^b ~ |i ~ P-el 'k ~ \h ~ /^^l ~^ ^) — "~n^} — 
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^ ^ ln( — ) ^ ^ ln( — ) 

'^BiP-t, + ^) < , ^b(/^£, l^e — ^) < } for £ = 1, • • • , s. Making use of these facts 
and Lemmas [75] and [76l we have 

s f 1 ('A'l 1 



1=1 



" I ln( — ) 



from which Theorem 1331 follows. 



1.20 Proof of Theorem | 

We need some preliminary results. 

Lemma 77 Let X„, = '"^^ — where Xi, ■ ■ ■ ,Xn are i.i.d. random variables such that < Xi < 
1 andE[Xi] = n e {0,1) for i = 1, ■ ■ ■ ,n. Then, Pr{X„ > /x, ^(X„,/x) < } < a /or any a > 0. 

Proof. For simplicity of notations, let Fj^ (z) = Pr|X„ < z}. By Lemma [H we have that 
{Xn. > = {^n > fJ-, — ^■'^P ('^•^ (^n)A*))}- Therefore, 

|X„ >fi,^ (X„, Ai) ^} = >fi,^ (X„, m) < PxMn) < exp (n^ (X„, ^i)) | 

C {i^-jX„)<a} 

and thus Lemma [77l follows from Lemma [2j 



□ 



Lemma 78 Let Xn = — ^, where Xi, ■ ■ ■ , Xn are i.i.d. random variables such that < < 
1 and E[Xi] = fie (0,1) fori = !,■■■ ,n. Then, Pr{X„ < ^, ^ (X„, /x) < ii^} < a for any a > 0. 

Proof. For simplicity of notations, let {z) = Pr{X„ > z}. By Lemma [H we have that 
{Xn < /w} = {Xn < fJ-, Gj^^{Xn) < cxp [n^ (X„, //))}. Therefore, 

|X„ <fi, .Jf (Xn,fi) < ^1 = < (Xn,^^) < Gx„(XO < exp (n^ (X„, /x)) | 

and thus Lemma [751 follows from Lemma [2j 

□ 

Now we are in a position to show Theorem I34[ By a similar method as that of Lemma [TOl we 



can show that — 2ln[^ ) ^ ^ ^^^^ event . By a similar method as that of Lemma 
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ttH we can show that - f )' ^ i + 51^} ^ {^b (/if , /if + e) < -^b (/if, Me - < 



-} for i = 1, • • • ,s. Therefore, by a variation of the argument for Theorem 1331 and using 
Lemmas 1771 and 1781 we have Pr{|/i — ^| > e} < 5, from which Theorem 1341 fohows. 



1.21 Proof of Theorem 



By a similar method as that of LemmalSTl we can show that {.^b(Ps7-^(Ps)) ^ ^'^f^^ , -^BiPsT^^iPs)) 



< 



} is a sure event. By Lemmas [75] and [76| we have 



" \ \n( — ) 

Pr{|/i-M|>£} < 5^Pr<^/i>'^(/i,), ^B(/i£,^(/if)) < 



e=i 



< ^PrL>M£, ^B(/if,M) < i +y]PrL< Mf, ^B(Mf , /i) < i < 



from which Theorem |38J follows. 



1.22 Proof of Theorem 



By a similar method as that of Lemma [69l we can show that {Ds = 1} is a sure event. By 
a similar method as that of Lemma [68| we can show that {£>£ = 1} c {^BiP-e^^iP-e)) — 
^BiPii-^iPe)) < ^'^nP } for £ = 1, • • • , s. Therefore, by a variation of the argument for 
Theorem and using Lemmas [77] and [75] we can establish Theorem [ 



J Proofs of Theorems for Estimation of Poisson Parameters 
J.l Proof of Theorem I 



First, we shall show statement (I). Let < < 1 and r = inff>o assumption that 



r > 1, we have that there exists a number £' > maxjr, r H — ^ + -r^l such that ^^^^ > for 

' L 5 ' 1 ln2 J ne 2 

any £ > i'. Noting that 

, 2 (£+l-r)ln2^1n(C<5) _ 2 / 1 \ 

'"(C-S^) r + 1 (£-T)ln2-ln(C(5) r + 1 I ^ _ ^ _ MC£) i 

\ In 2 / 

for £ > f and that lilKM = M£^! — ) ^ > ./#p(-, - + e) as £ 00, we have that there exists an 
integer k greater than £' such that ^p(-, - + e) < '"^'''^"^ for all i > k. For i no less than such k, 
we claim that z > ^ if ^p{z,z + e) > '"^^^"^^^ and 2: G [0, 00). To prove this claim, suppose, to get 
a contradiction, that z < ^. Then, since ^p{z, z + e) is monotonically increasing with respect to 
z > 0, we have J^p{z, z + e) < ^p(-, - + e) < lllKM^ which is a contradiction. Therefore, we have 
shown the claim and it follows that {.-#p(^, ^ + e) > lilKM} c > A} for ^ > k. So, 



Pr{Z >i}<Vr{^pi^^,^+ e\ > < Pr > ^ j < cxp {-cue) , 

ne J ne } [ne rj ' 
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where c = — ^p(^,A) and the last inequality is due to Chernoff bounds. Since Pr{Z > i} < 
exp(— cn^) and — )• oo as £ — )• oo, we have Pr{Z < 00} = 1 or equivalently, Pr{n < 00} = 1. This 
completes the proof of statement (I). 

To show statement (II) of Theorem we can use an argument similar to the proof of 
statement (II) of Theorem [23J 

To show statement (III) of Theorem we can use an argument similar to the proof of 
statement (III) of Theorem [23l 

To show statement (IV) of Theorem we can use an argument similar to the proof of 
statement (IV) of Theorem [23] and make use of the observation that 



Pr 



A- A 



Xf-X 



I A} 



>e|A} = ^Pr{ Af-A >£, Z = £| a}+ ^ Pr{ 

e=i e=e*+i 

t t I* 

< ^Pr{Z = £ I A} + ?7 < ^PrjAf < Z£ | a| + 77 < ^ exp(n£^p(zf , A)) 



i=i 



1=1 



To show statement (V) of Theorem we can use an argument similar to the proof of 
statement (V) of Theorem | 



J. 2 Proof of Theorem [45] 

Theorem [45] can be established by using a method similar to that of Theorem [27] based on the 
following preliminary results. 

Lemma 79 Let e > 0. Then, ^■p{z^ z + e) is monotonically increasing with respect to z > 0. 
Proof. Note that ^p{z, z + e) = -e + zln (^) and 



d^p{z, z + e) 



In 



z + e 



In 1 



dz \ z J z + e \ z + e 

where the inequality follows from ln(l — x) < —x, Vx G [0, 1 

Lemma 80 lim£_>.o SI=i n£e~^'^ for any c > 0. 



> 0, Vz > 



z + e 



□ 



Proof. Lemma [80] can be shown by a similar method as that of Lemma [T4l 



□ 
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Lemma 81 If e is sufficiently small, then the following statements hold true. 

(I) : For £ = 1, - ■ ■ , r, there exists a unique number zg S [0, oo) such that un = 

(II) : zg is monotonically increasing with respect to I no greater than r. 

(III) : lim^^Q Z£ = \*Cr-e, for 1 < i < t, where the limit is taken under the restriction that 
i — T is fixed with respect to e. 

(IV) : {De = 0} = {A, > z^} for £ = 1,- ■ ■ , r. 

Proof. Lemma [81] can be shown by a similar method as that of Lemma [JHl □ 
Lemma 82 Define £s = t — j\, where j\ is the largest integer j such that Cj > Then, 

4-1 r 

hm V Pr{i:)£ = 1} = 0, hm V niFr{Di = 0} = (144) 

e=l ^=4+1 

for A G (0, A*). Moreover, hm£_^o "4 P'^iDe, = 0} = for X e (0, A*) such that Cj^ > ^. 

Proof. For simpHcity of notations, let bg = lime_5.o Z£ for 1 < £ < t. 

First, we shall show that (jl44p holds for A € (0, A*). By the definition of £s, we have bg^-i = 
A*Ct— 4+1 = X*Cj^+i < A. Making use of the first three statements of Lemma EU we have that 
ze < '^'''''i''^ < A for all £ < — 1 if e is sufficiently small. By the last statement of Lemma [HTl we 



2 

have 



Pi{De = 1} = Pr{A, < z,} < Pr |a, < A±^| < cxp (^n,^p (^A±^, 

for all ^ < — 1 provided that e > is small enough. Since ^^'^^'-^ is independent of e > 0, we 
have lim£_j.o J21l(^ Vt:{Di = 1} = as a result of Lemma [80l 

Similarly, it can be seen from the definition of £e that ft^^+i = A*Ct— ^^-i = A*Cj^_i > A. 
Making use of the first three statements of Lemma 1811 we have that zi > > A for 4 + 1 < 

£ < r if e is sufficiently small. By the last statement of Lemma [5T1 we have 

Pr{£>, = 0} = Pr{A, > ze} < Pr |a, > A±^| < cxp (^m^p (^^±^i±l^\ 

for 4 + 1 ^ ^ ^ provided that e > is small enough. Therefore, we can apply Lemma [80] to 
conclude that linie_j.o ^21=1^+1 Pi'{-Df = 0} = 0. 

Second, we shall show that lim£_5.of^4 Pi'i-D^ = 0} = for A G (0, A*) such that Cj^ > 
Clearly, bi^ = X*Cr-e^ = ^*Cj^ > X. Making use of the first three statements of Lemma [ST] we 
have ze^ > > A if e is sufficiently small. By the last statement of Lemma [HT] we have 

Pr{D,^ - 0} = Pr{A,^ > z,J < Pr|A,^ > ^^^| < cxp (uf^^p (^^^^,A 

for small enough e > 0. It follows that linie^o Pr{-D^^ = 0} = 0. 

□ 

Lemma 83 lims_,o E^r+i "-^ = £} = for any A G (0, A*). 
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Proof. Recalling that the sample sizes ni,n2,--- are chosen as the ascending arrangement of 
all distinct elements of the set defined by ()29p . we have that 



rii 



Cr-i HCS) 
(A*,A* +e) 



1,2,. 



for small enough e > 0, By the assumption that inf^^^ -4^ = 1 + p > 1, we have that 



for small enough e > 0. So, there exists a number e* > such that 

ne^P {X* , X* + e) < {1 + pY~^~^ ln{C6), £ = t + 1,t + 2,- ■ ■ 

for any e G (0,e*). Observing that there exist a positive integer k* such that (1 + /))^"'^"-'^ ln(C'^) < 
ln(^(5) — — r) ln2 = ln((^(5f) for any i > t -\- k* , we have that there exists a positive integer k* 
independent of e such that .^p(A*, A* + e) < ^^^^ for ^ > r + k* and < e < e*. Note that 
^p{z,z + e) is monotonically increasing with respect to z € (0,oo) as asserted by Lemma [79l 
For i > T + K* and < e < e*, as a result of > ^p(A*, A* + e), there exists a unique number 



2;^ G [0, oo) such that .J^p{ze,ze + e) ~ "^^^ " > >^p(A*, A* + e). Moreover, it must be true that 
ze > X* for £ > T + K* and e G (0, e*). Therefore, for small enough e G (0, e*), we have 

oo r+K* oo 

^ Pi{l ^e} = ^ ni Pr{l ^ £} + ^ ni Pr{l = £} 

e=T + l t = T+l t=T + K* + l 

r+K* oo 

< ^ n^Pr{I3, = 0}+ "fPi-{-Df-i =0} 

r+K* oo 



^ n^Pr{I3, = 0}+ Yl ^f+i Pi-{-Df = 0} 

?=T+1 £=t+k;* 



< fc*(l + p)''*n^Pr{D^ = 0} + (1 + p) ^ n£Pr{Df = 0} 

OO 

< k*{l+-pf UrPliXr > Zr] + {l+Jl) ^ PrjAf > } 

< fc*(l + p)'=*n,Pr ^A, > + + "£Pr{A£>A*} 



( / A + A* 

< +^)^*n^exp iur^p i — - — , A 

oo 

+ (l + p) Y n-f exp(n£^p(A*, A)) -> 

as e — 0, where we have used the assumption that supjg^ = 1 + p < oo. This completes the 
proof of the lemma. □ 
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J. 3 Proof of Theorem [47] 



To show statement (I) of Theorem ITTl we can use an argument similar to the proof of statement 
(I) of Theorem [53 

To show statement (II) of Theorem Wl[ we can use an argument similar to the proof of 
statement (II) of Theorem [23j 

To show statement (III) of Theorem [JTl we can use an argument similar to the proof of 
statement (III) of Theorem [231 

To show statement (IV) of Theorem Wl\ we can use an argument similar to the proof of 
statement (IV) of Theorem [23] and make use of the observation that 



Pr 



A- A 



Xf-X 



> eX, I 



I A} 



> eA I a} = ^ Pr { Af - A > eA, « = £ I a} + ^ Pr { 

e=i i=i*+i 
r 

< ^Pr{ Af- A > eA, « = £ I a} +?7 

e=i 

t t t 

< ^ Pr = ^ I A} + ?7 < ^ Pr |Af > I a| + 77 < ^ exp(n£^p(zf , A)) + 77. 



e=i 



1=1 



Pr 



A- A 



> eA I A ^ < Pr 



A- A 
Ai - A 
Ai - A 



To show statement (V) of Theorem W7\ we can use an argument similar to the proof of 
statement (V) of Theorem [23] and make use of the observation that 

> eA, Z = 1 I a} + Pr I A - A > eA, Z > 1 | a} 

< Pr <! Ai - A > eA I a} +Pr{Z > 1 I A} 

< Pr I Ai - A > eA I a} + Pr I Ai < zi I a} 

< 2exp(ni^p((l + e)A, A)) + exp(ni^p(2;i. A)). 

J. 4 Proof of Theorem [48] 

Theorem [l8] can be established by using a method similar to that of Theorem [27] based on the 
following preliminary results. 

Lemma 84 lime_j>o X]£=i ra^ e"""^^ for any c > 0. 



Proof. Lemma [8^ can be shown by a similar method as that of Lemma [TU 



□ 



Lemma 85 If e is sufficiently small, then the following statements hold true. 

(I) : For i = 1, - ■ ■ ,T, there exists a unique number Z£ G [0, 00) such that n£ = '°^^'^^L . 

(II) : zg is monotonically decreasing with respect to i no greater than r. 

(III) : lim£_s.o = ^^-^ for I < i < t, where the limit is taken under the restriction that £ — t 
is fixed with respect to e. 

(IV) : {De = 0} = {A, < z,} for £ = 1, ■ ■ ■ , r. 
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Proof. Lemma [85] can be shown by a similar method as that of Lemma H8l □ 



Lemma 86 Define (-e = t — j\, where j\ is the largest integer j such that Cj > ^. Then, 



limS^ nePv{Di = l} = 0, Km V PriZ)^ = 0} = (145) 

1=1 l=le+l 

for A G (A', A"). Moreover, Hme^^o^^^ Prj-D^ = 0} = /or A G (A', A") such that Cj^ > y. 

Proof. For simplicity of notations, let bi = limg^o for 1 < i < t. 

First, we shall show that (jl45p holds for A G (A', A"). By the definition of ig, we have 
bf _i = 7-!-^ = — > A. Making use of the first three statements of Lemma [851 we have 



that zi > > A for alU < 4 - 1 if 

£ is sufficiently small. By the last statement of Lemma 



we have 

Pi{De = 1} = Pi-{Xe > ze} < Pr |a, > ^^y^j < cxp (ne^p (^ ^^^'°'\ a 

for all £ < ig — 1 provided that e > is small enough. Since '^"^''^'''^ is independent of e > 0, we 
have lim£_).o X^t^^ Pi'{-D^ = 1} = as a result of Lemma [ 

h - A' _ 



Similarly, it can be seen from the definition of 4 that ft^^+i = cr^i — ^ ~ ^^.^ ^ < A. Making 



use of the first three statements of Lemma \85\ we have that Zi < — |^ < A for ie + l^ ^ i ^ t 
£ is sufficiently small. By the last statement of Lemma [85l we have 

Pr{Df = 0} = Pr{Af < < Pr (a^ < ^±}h±l \ < cxp (mJip f ^-t.^^, A 



for 4 + 1 ^ ^ ^ provided that e > is small enough. Therefore, we can apply Lemma [85] to 
conclude that linie_i.o X]I=^^+i Prj-Dc = 0} = 0. 

Second, we shall show that Xxm-g-s^^nn^ Pr{Z)^^ = 0} = for A G (A', A") such that Cj^ > ^. 
Clearly, bi = — = 7^ < A. Making use of the first three statements of Lemma [85l we have 

< ^ < A if £ is sufficiently small. By the last statement of Lemma [551 we have 
Pr{r>,, = 0} = Pr{A,, < z,J < Pr|A,^ < < cxp (^n^^p (^^,A 



for small enough e > 0. It follows that linie^o Pr{-Df^ = 0} = 0. 



Lemma 87 limg^^o YT=t+i = ^} = for any A G (A', A")- 



□ 
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Proof. Recalling that the sample sizes ni,n2,--- are chosen as the ascending arrangement of 
all distinct elements of the set defined by ([30]) . we have that 



Hi 



n, > (1 + pY 



Cr-t HC5) 

^p(a',^) 
assumption 

.-1 HCS) 



1,2,... 



for small enough e G (0, 1). By the assumption that infjgz -^-^ = 1 + p > 1, we have that 



r + l,T + 2, 



for small enough e G (0, 1). So, there exists a number e* G (0, 1) such that 



l + e 



for any e G (0, e*). Observing that there exist a positive integer k* such that (1 + pY~'^~^ InCC*^) < 
ln(i^(5) — — r)ln2 = ln{(^5e) for any £ > r + k*, we have that there exists a positive integer 
At* independent of e such that ^p(A', ^) < ^^^^^f^ for £ > r + k* and < e < e*. Note that 
z[j4- — ln(l + e)] is monotonically decreasing with respect to z G (0,oo). For 



'' l+e' 



> T + K* and < e < £*, as a result of ^^^''^^^"^ > ^p{\' , j^), there exists a unique number 

— m > -#p(A', j^)- Moreover, it must be true that < A' 



G [0, oo) such that J^p{zt, = > ^p(A', ^ 

for 1>T + K* and e G (0, e*). Therefore, for small enough e G (0, e*), we have 



^ n^Pr{« = ^} = ^ 7i^Pr{i = £}+ ^ Pr{« = ^} 

r+K* oo 

< ^ n,Pr{r>, =0}+ ^fPr{£>f-i =0} 

r+K* oo 

= "^Pr{-Dr=0}+ ^ n£+iPr{Df = 0} 

oo 

< r(l + p)''^*n^Pr{D^ = 0} + (1 + p) ^ n£Pr{Df = 0} 



< fc*(l + p)''^*?vPr{Ar < Zr} + (1 +p) ^ n^Pr{A^<Zf} 

r A' + A 1 °° - 

< r(l + p)'=*n,Pr<^A,< ^ + (l + p) ^ n,Pr{A,<A'} 



— T + K* 



< /c* (1 + p) n-r GXp llr^^ 



AH- V 



-(1 + p) ^ exp(n£.^p(A', A)) ^ 



as e — )■ 0, where we have used the assumption that supjg^ = 1 + p < oo. This completes the 
proof of the lemma. □ 
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J. 5 Proof of Theorem [49] 

We need some preliminary results. The following results, stated as Lemma [88l can be derived 
from Chernoff bounds. 

Lemma 88 5p(0,fc, nA) < exp(n.^p(^, A)) forO < k < n\. Similarly, Sp{k,oo,nX) < exp(n^p(^, A)) 
for k > nX. 

Lemma 89 ^p(A - e, A) < ./#p(A + e, A) < for any e G (0, A]. 

Proof. In the case of e = A > 0, we have ^p{X + e, A) = e — 2e In 2 > — e = ^p{X — e, A). In the 
case of < e < A, the lemma follows from the facts that ^p(A + e, A) = ^p(A — e, A) for e = 
and ^[^p(A + e, A) - ^p(A - e, A)] = In -j^rz^ > for any e G (0, A). To show ^p(A + e,X) < 
for any e G (0, A], note that ^p(A + e, A) = e + (A + e) In ^ < e + {X + e) x ^ = 0. This 
completes the proof of the lemma. 

□ 

Lemma 90 Let e > 0. Then, ^p{z, z — e) is monotonically increasing with respect to z > e. 

Proof. Note that ^p{z, z — e) = e + zln (^7^) and 

d^p(z,z-e) / z - e\ e e \ e 

= In + = - In 1 + + > 

oz \ z J z — e \ z — e J z — e 

where the last inequality follows from ln(l + x) < x, Vx G [0, 1). 

□ 

Lemma 91 Let < e < 1. Then, J^p{z, ■^) < .Mp{z, and ■^.J(p{z, j^) < ■^y£p{z, < 
for z > 0. 

Proof. Note that ^p{z,j^) - ^^piz,-^) = z g{e) where g{e) = + + In(Y^). Since 
g{0) = and ^ = (jzpjr > 0, we have g{e) > for < e < 1. It follows that ^piz, < 
^piz, jf^). 

Using the inequality ln(l — x) < — x, Vx G (0, 1), we have ^^p(z, ~ + ln(l - j^) < 0. 
Noting that ^[^p{z, j^) - .£p{z, -^)\ = g{e) > 0, we have -§^^p{z, j^) < -§^^p{z, j^) < 0. 

□ 

Lemma 92 Pr{^p(A„ ^(A^)) < .^p(A„^(A,)) < '-^} = 1. 
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Proof. For simplicity of notations, we denote X* = j^. By the definitions of ^{Xg) and (A^), 
we have that, in order to show the lemma, it suffices to show 

(xs, > A. > A* - 6«| = 0, (146) 

^p(A„ A, + Ea) > Xs<X'-ea]=i/}, (147) 

>i^, A, >A* + 6j=0, (148) 



"'"i I - -a 7-, . I' 

\ I + Sr I n 

^P(A„ A, - £a) > A, < A* + e4 = 0. (149) 



By the definition of Ug , we have Ug > ^^(vjj,^ a*) — ^p(\*+e a*) ■ ^^"^ assumption on and 
Er, we have < < A*. Hence, by Lemma[89l we have (A* — Eq, A*) < ^p (A* + Ea, A*) < 
and it follows that 

ln(C<5) 



rig 



> Jiv (A* + Ea, A*) > Ji^ (A* - Ea, A*) . (150) 



By (HSOD, 



|^p|^A„^^j >^^, A, >A*-ea| C |.^p I^A^^^j >^p(A*-ea,A*), A, >A*-ea|. 

(151) 

Noting that Jt-p (A* — £«, A*) = Ji-p (^A* — Eq, and making use of the fact that .-#p(z, 

is monotonically decreasing with respect to z G (0,oo) as asserted by Lemma EU we have 

|.^P ^A„ > ^p (A* - e„ A*) I = {Xg < A* - ej. (152) 

Combining (fTST]) and (fT52]) yields (fTi6]) . By (fTSO]) . 

|^p(A,,A, +£a) > A, < A* -£a| C |.^p(A,,As +£a) > ^P (A* - A"^) , As < A* -£a} ■ 

(153) 

By the assumption on Ea and e,-, we have A* — > 0. Recalling the fact that ^p^z^z + e) is 
monotonically increasing with respect to z G (0, oo) as asserted by Lemma [79l we have that the 
event in the right-hand side of ([153]) is an impossible event and consequently, (jl47p is established. 
By ([150]), 

l^P [xg, > i^^, A, > A* + £„| = |.^p [xs, > .^p (A'^ + £a, A*) , A, > A^ + £a| . 

(154) 

Noting that ^p (A* + Ea, A*) = ^p ( A* + Eq, Kri^ ) and making use of the fact that Jip{z^ -r^) 



is monotonically decreasing with respect to z G (0,oo) as asserted by Lemma [9T] we have 

l^p (a.,3-^J >^p(A^ + ea,A'^)| ={A, <A* + e4. (155) 
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Combining (fTSil) and (fTSSD yields (fTiSD . By (fT50D . 

|^p(A,,A, -£a) > A. < A*+£a| C { .^p ( A. , A, - £a) > (A* + Ea, A^) , A, < X* + Ea] ■ 

(156) 

Recalling the fact that ^p{z,z — e) is monotonically increasing with respect to z € (e, oo) as 
stated by Lemma [90l we have that the event in the right-hand side of (I156p is an impossible event 
and consequently, (jl49p is established. This completes the proof of the lemma. 

□ 



Lemma 93 Pr 



> Er \ X 



I < (5 /or A G [A, oo). 



Proof. Note that 
A- A 



Pr 



A 



> £r \ X 



> Er, I 



A < 



i=l 



> Er \ X 



< ^ [exp(?i^^p(A + XEr, A)) + exp(?i^^p(A - XEr, A))] (157) 

s 

< 2^exp(n^^p(A(l + A)) 

where (jl57p follows from Lemma [3T1 Since lim;v^o ^p(A(l + Er),X) = and lim;^_^oo -#p(A(l + 
Er), A) = — OO, there exists a unique number A > such that X]|=i exp(n£^p(A(l + e^), A)) — 



2- 



Finally, the lemma is established by noting that ^p(A(l + Er),X) is monotonically decreasing 
with respect to A > 0. 

□ 



Now we are in a position to prove Theorem |39j The second statement of Theorem [39] is a 
result of Lemma [93j 

If the multistage sampling scheme follows a stopping rule derived from Chernoff bounds, then 
{Ds = 1} is a sure event as a result of Lemma [92l Note that ^p{z, A) = inft>o e~*^ E[e*'^''] and 
that Xi is a ULE of p for £ = 1, • • • ,s. So, the sampling scheme satisfies all the requirements 
described in Corollary [H from which Theorem [39] immediately follows. 

If the multistage sampling scheme follows a stopping rule derived from CDFs, then, by Lemmas 
[88] and [921 we have 

Pr{Gx^(A«,if(A,)) < C<5.} = Pr{l - ^p(/v. - l,n,J^(A,,)) < CS} 

> Pr{n,.^p(A„.5^(A,)) < ln(C<5)} = 1, 
Pr{Fj^^{\s,^{\s))<CS,}^Pv{Sp{Ks,ns^{\s))<CS}>Pv{ns^p{\,,^{\s))^ 

and thus PrlFr (A,,^(A,)) < C^s, (A^,^(AJ) < C^J = 1, which implies that {D, = 1} 
is a sure event. So, the sampling scheme satisfies all the requirements described in Theorem [21 
from which Theorem [39] immediately follows. 
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J. 6 Proof of Theorem [5T] 

We need some preliminary results. 



Lemma 94 limj^_>.o X^^^i ni e "■^'^ = for any c > 0. 



Proof. For simplicity of notations, define A* = |^ as before. By differentiation, it can be shown 
that xe~^'^ is monotonically increasing with respect to x E (0, ^) and monotonically decreasing 

with respect to x G (i,oo). Since the smallest sample size ni > ig greater than ^ for small 
enough e > 0, we have that ni e""**^ < sni e~'^^'^ if > is sufficiently small. Let 



p ~ inf£>o — 1. Observing that 



s < 1 



In 



-^p(A*+ea,A*) 

ln(l + P) 



In 



< 1 



.-«rp(A*+e„,A*) 

ln(l + p) 



In J, 



and ni > — we have 



In 



.^p(A*+e„,A*) 
lll(l + p) 



exp 



for small enough Ea > 0, where A{sa) = ^ exp — f-^) and -B(ea) 



c lii(l + p) 



S(ea) 



■ exp 



Noting that lim^._5.oo xe~^' = and that ^-7-^ — > 00 as Eq — > 0, we have lim£^_j.o ^(ea) = 0. Now we 
show that lim£^_j.o -B(ea) = 0. Using Taylor's expansion formula ln(l + x) = x — ^ + ^ + o(x^), 
we have 



23 2 
^ ^ u o(e^) = — + we^ + o(e^] 



where zu = Hence, 



In 



In- 



^ + roe3 + o(e3) 



ln(2A*) + In — + In 



2A 



1 - 2X*u7ea + o{ea) 



1 



ln(2A*) + In — + 2X*ujea + o{ea) 



and 



In 



(158) 



Using pSSp and the observation that 



[2X*w + 0(1)] exp 



cln^' 



ln(2A-) / clni^ 
0(1), exp ^ 



ln(2A" 



cln 



^1^^ exp/^ 



0(1) 
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In — / c In — \ 1 

we have B{ea) = o(l) H — j^^expf r^j- Making a change of variable x = — and using L' 

Hospital's rule, we have 

( . x\nx 1 + liia; r ^ n 
iim B^£a) = lim — — = lim — — - — —^x = hm ^ = (J. 



a;— >oo 



In — 

Therefore, < linisup^^_^o 

ni e-"*= < i lime„-yo A(£a) + x linie^^o B(£a) = 0, which 

implies that lime^_>o X^|=i e""^"^ = 0. This completes the proof of the lemma. 



□ 



Lemma 95 If Sa is sufficiently small, then the following statements hold true. 

(I) : For 1 < £ < s, there exists a unique number Z£ £ [0, X* — Sa) such that = z^^^^fl^+T^' 

(II) : For 1 < i < s, there exists a unique number E (A* + ea, oo) such that = — V- 

(III) : zi is monotonically increasing with respect to I; yi is monotonically decreasing with 
respect to i. 

(IV) : lim^^^o Zi = \*Cs-e and lim^^^oyi = ^ , where the limits are taken under the con- 
straint that Y' ^'^^ s — I are fixed with respect to Sa ■ 

(V) : Let is = s — jx- For A G (A*, oo) such that Cj^ = r(A), 

Zf — X 
lim ' ^ = 0. 

Er-S'O e^A 

For X G (0, A*) such that Cj^ =r{X), 

lim^^ = ^fA_ 

Ea 3 \A* 

(VI) : {De = 0} = {zi < < y^} forl<i<s. 
Proof of Statement (I): 

By the definition of sample sizes, we have > ^p(0,ea) and 



< — 2 — < -^r- 

for sufficiently small £a > 0. By (jl59p . we have 



^p{X* + £a,X'') 



(159) 



In(C^) 2 .^p{X* + ea,X*) 2^piX* -Ea, X*) .J^p(A* + e,,, A*) 



ne ' ' '\1 + Ci nej ^p{X* - Sa, X*) 1 + Ci ne 

Noting that 



^. ^p(A^ + ea,A*) ^ ^p{X* + ea,X*) ^ 

lim — = 1, lim = 0, 

£a^o ^p(A* — Eq, a*) sa^o ni 



we have that ^^^^^ < .^p(A* — A*) for small enough £a > 0. In view of the established fact that 



^p(0, Ea) < ^^n^^ < -^P i^* ~ A*) and the fact that ^p(z, z + Sa) is monotonically increasing 
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with respect to z > as asserted by Lemma [79l invoking the intermediate value theorem, we have 
that there exists a unique number zn S [0, A* — Sa) such that .#p(z^, z^+Sa) = ^"^"^^^ , which imphes 
Statement (I). 

Proof of Statement (II): By (jl59p . we have 



nil \ 1 + Ci nf / \ 1 + Ci / \ l + Er J Ui 

Noting that lim^^^o = 0, we have that < ^p(A* + e.^, Ali^) for smah enough 

ffa > 0. In view of the estabhshed fact that ^^^^ < .-#p(A* + Sa, ^j^r^) and the fact that 
-^piz, 1^^) is monotonically decreasing to — oo with respect to z G (0, oo) as asserted by Lemma 
\9T\ invoking the intermediate value theorem, we have that there exists a unique number yg S 
(A* + Ea, oo) such that ^p{yi, = ^^n^ ; which implies Statement (II). 

Proof of Statement (III): Since ng is monotonically increasing with respect to ^ if > 
is sufficiently small, we have that ^p{zi,ze + Sa) is monotonically increasing with respect to i 
for small enough Sa > 0. Recalling that ^p{z, z + Ea) is monotonically increasing with respect to 
z > 0, we have that Z£ is monotonically increasing with respect to i. Similarly, ^p{yi, j^^) is 
monotonically increasing with respect to i for sufficiently small Ea > 0. Recalling that ^p{z, ) 
is monotonically decreasing with respect to z > 0, we have that y^ is monotonically decreasing 
with respect to L This establishes Statement (III). 



Proof of Statement (IV): We first consider \\m.i,^_^Q zg^. For simplicity of notations, define 

hi 

X* 



bi = X*Cs-e for £ < s. Then, it can be checked that -fe- = Cs-e and, by the definition of sample 



sizes, we have 

be ^p{ze,ze + Ea) ^ 1 ^, Cs^e ln{C6) 

X* ^p{\* + Ea,X*) rii J^p{X* + Ea,X*) 

for £ < s. 

We claim that ze > 9 for 9 G (0, be) if Ea > is small enough. To prove this claim, we use 
a contradiction method. Suppose this claim is not true, then there is a set, denoted by S^^, 
of infinitely many values of Ea such that ze ^ 9 for any Ea G Se^. By (fT60]) and the fact that 
^p{z,z + Ea) is monotonically increasing with respect to z > as asserted by Lemma \79\ we 
have 

be ^p{ze,ze + Ea) , , ^ be ^p{9,9 + Ea) be , 



A* ^p(A* + Ea, X*) ' ' - A* ^p(A* + Ea, A* 

for small enough Ea G Ss^, which implies ^ < 1, contradicting to the fact that ^ > 1. This 
proves the claim. Now we restrict Ea to be small enough so that 9 < Z£ < X* . Since zi is bounded 
in interval {9, X*), we have ^p{ze, ze + Ea) = — ^a/ (2-z^) + o{Ea) and by (|160p . we have 

be ^ -el/{2ze) + o{eI) _ 
A^"-.^/(2A^) + o(.D" ^'^ ^' 

which implies ^ = 1 + o(l) and thus lim^^^o-^^ = be- 
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We now consider lime^_!.o ye- For simplicity of notations, define ae 



Cs 



for 1 < ^ < s. Then, 



it can be checked that — = Cs-t and, by the definition of sample sizes, we have 



A* 



l+e, 



) 



1 



Cs-i HC5) 



l + o(l). 



(161) 



a£ ^p(A* + A*) rii .^p(A* + A*) 

We claim that < 9 for 6 G (a^,oo) if > is small enough. To prove this claim, we 
use a contradiction method. Suppose this claim is not true, then there is a set, denoted by 5^^, 
of infinitely many values of Er such that ye > 9 for any Er G Se^- By (jl6ip and the fact that 
-) is monotonically decreasing with respect to z G (0, oo) as asserted by Lemma [911 we 



have 



A- -^Ay^,-^) 

ae J^-p{\* + Ea, A* 



1 + 0(1) > 



A* 



l+er 



ae ^p(A* + Ea, A*) 



ae 



+ o(l) 



for small enough Er G S^^, which implies ^ < 1, contradicting to the fact that ^ > 1. This 
proves the claim. Now we restrict Er to be small enough so that X* < ye < 9 . Since ye is bounded 
in interval (A*, 9), we have ^p{ye, jr—) = —s'lye/'^ + o(e^) and by (|161|) . we have 



which implies 



y -Elye/2 + o{eI) 
ae -El/{2X*) + o{eI) 

o(l) and thus lim^^^oy^ = o^i- 



1 + 0(1), 



Proof of Statement (V): 

We shall first consider A G (A*,oo) such that Cj^ = 4^. Let Ve be a function of e G (0,1) 
such that is bounded from above by a constant independent of e. Then, by Taylor's series 
expansion formula, we have 



1 



l + e ' 



A ln(l + e) = e V'Jl - e + + o{e^)] - tP, 



,2 3 



+ -^+o{e^) 



2 3 

for e G (0, 1). By the definition of sample sizes, for small enough e^, there exists 
such that 



HC5) 



^pize^,zej{l + Er)) 



C. 



HC5) 



A* 



HCS) 



A ^p{X*,X*/{l + Er)) 



(162) 
G (A*,oo) 

, (163) 



^p(A*,AV(l + er)) 

from which we can use an argument similar to the proof of Statement (III) to deduce that ze^ is 
smaller than 9 for G (A, oo) if > is small enough. Hence, by (1162^ and (jl63p . we have 



1 + 0{Er) 



A ^p{x\x*/li+er)) _ A* ^p{ze^,zej{l + Er)) _ y 



+ ^ + o(g^) 



A ^p(A^AV(l + ^.)) A _£^ + M^ + o(e3) 



and consequently, 



1 + o{Er) 



yze, 
X X* 



4erA* 



+ o(gr) 
+ o{Er) ' 
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which imphes that 



X{y-^]+o{er), 



i.e., Zi^ (l - ^) = A (l - ^) + o{er), i.e., 24 = A + o{er)- It follows that linie^^o ^ 



and 



thus 



lim 



6 lim = 0. 

er-!>0 SrA 



Next, we shall now consider A G (0, A*) such that Cj^ = Let V'e be a function of e G (0, 00) 
such that is bounded from above by a constant independent of e. Then, by Taylor's series 
expansion formula, we have 



f f2 3 



■ + (164) 



2^e ' 3^2 

By the definition of sample sizes, for small enough e^, there exists z^^ £ (0, A*) such that 



ln(C<5) 



^p{ze,,zg^ +ea) 



A 



ln(C5) 



A*.^p(A^A* 



(165) 



^p(A*,A* + eJ 

from which we can use an argument similar to the proof of Statement (III) to deduce that zi^ is 
greater than 9 for 9 G (0, A) if > is small enough. Hence, by (jl64p and (jl65p . we have 

-3^ 



1 + 0{£a) 

and consequently. 



A ln(C<5) 
A* .^p(A*,A*+£, 



-) _ X ^p(z4, +ea) _ A 24 + 3^ + 



ln(C5)_^ 



X- .^p{X\X* + Sa) A-_^ + _£L^ + o(e3) 



1 + o(ea) 



l-||- + o(eJ ^ 



which imphes that — = 1 - |ff + o{ea), i.e.. 



2A 



3A* 3ze^ ^ -^f^e ea 



+ Zi J^^^^. So, we have 



lim^^ = ^fA_i|<o. 

ea-!>0 3 V a* 



Proof of Statement (VI) : By the definition of the sampling scheme, we have 
{Dt = 0} = (max{.^p(A,, A,), .^p(A,, A,)} > i^^, |A, - A*| < Sa 

(J |max{.^p(A£, A(,), ^p(A£, Af)} > i^^^^, A^ < A* - 



IJ <^ max{.^p(Af, A^), .^p(Af, A^)} > 



ni 



max < ./#p(A£, Af — £„), ./#p ( A^, 



1 - e. 



Af > A* + Ea 



>i^, |A,^A1<s. 



U|^p(A,,A, + e.)>^"(^^) 

Uppf 



In(C^) 



1 + er 



Af < A^ - Sa 

, Af > A"^ + ea 
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We claim that, 



max < ^p(A£, Xi — Eq), A^, 



1 — £r 



> , |Af - A I < £q 

ni 



.^p(Af , A^ + Ea) > il^i^, A£ < A* - £a }> = {z^ < Xi < X* - £a}, 



1 + £r 



> 



ne 



Xi> \* + Ea 



|a* +£a <Xi < yej 



(166) 
(167) 

(168) 



for 1 < i < s provided that is sufficiently small. 
To show ()166p . note that 

(1 + Ci)n, 1 + Ci 
'^i < 7^ < — — 



HCS) 



+ 1 



(169) 



from which we have 
ln(C<5) ^p(A* + ea,A* 



< 



Ui ^p{\* - ea,\* - Ea- Ea) \l + Ci 

Noting that 



.^p{X* - Ea,X* - Ea - Ea) 



^p{X* + Ea,\* 



lim 



^p{X* + Ea,X*) 



lim 



% + oiEl] 



2X 



ea^O J^piX* - Ea, X* - Ea- Ea) £a->0 ^ + o(e'^) 

2(a —Ea) ^ ^' 



and lime^_s.o •^'p^^ ^^'^■•^ ) = g, we have 



nil 



ln(C5) 



Eai X Ea Ea) 



(170) 



for small enough > 0. Again by (1169^ . we have 

2 



ln(C5) ^ J^piX* + EaA* 



ni ^p(A^ + e„,Ali^) Vl + Ci 
Noting that 



A* + gg ^ ^p(A* + ea,A* 
1-e, 



and lim, 



^p(A^ + ea,A*) 
lim r— = lim ■ 

£a^o^p(A* + eg,^^) ea^o 

0, we have 
ln(C<5) 



^ + o(e^) 



'2{A*+£a) ^ I (l-£r)" 



< A* + Ea, 



1 — Er 



(171) 



for small enough > 0. Note that, for z £ [X* — Ea, A* + Ea], ^p{z,z — Ea) is monotonically 
increasing with respect to z and .^p{z, j^^) is monotonically decreasing with respect to z. By 
(fTTOll and (fTTT]) . we have ^^K^i < ^p(z, z^ej and < ^p(^z, ^) for any z G [A* -gg, A* + gg] 
if gg > is small enough. This proves (|166p . 
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To show (jl67p . let lo e {^p{Xi,'Xi + £a) > '"^'''^^ , < A* — Sa} and = \£{uj). Then, 



A^+^a) > '"f5^^ and A^ < X* — ea- Since zi G [0, A* — Eq) and ^p(z, 2;+ea) is monotonicany 



ni 



increasing with respect to z G (0, A* — Sa), it must be true that A^ > Z£. Otherwise if A^ < 
Z£, then ^p(A£,A£ + < ^p{ze,ze + Sa) = ^^^^^ , leading to a contradiction. This proves 
l^p{\i, Xi + Ea) > < A* - Ea} Q {zi < Xi < X* - Sa}. Now let w G {z^ < < A* - Sa} 

and Xg = X£{lj). Then, < A^ < A* — Sa- Noting that ^p{z,z + Ea) is monotonically increasing 
with respect to z > 0, we have that ^p(A^,A^ + Sa) > J^p{zi,zi + Sa) = '"^'^'^"^ , which implies 
{^p(Af , Xi + Sa) > A^ < A* - Sa} 2 {z£ < A<. < A* - ej. This establishes (fWll . 

To show (fTBHD . let w e {^p(A£,^) > i^g^), A^ > A* + Sa} and A^ = A^(tj). Then, 
•^pi^i^i^) > ^'^nP and Xi > X* + £a- Since yi G (A* + ea,co) and .y£p{z,j:^) is monoton- 
ically decreasing with respect to z > 0, it must be true that Xi < ye- Otherwise if A^ > ye, 
then ^p(Xi, < ^p{ye, jf^) = ^2^, leading to a contradiction. This proves {.^p(A^, > 
Xf > X* + ea} c {X* + ea <Xe< Vi}. Now let uj G {X^ + Sa <Xi< ye} and A^ = Xe{io). Then, 
X* + Ea < Xe < ye- Noting that ^p(z, is monotonically decreasing with respect to z > 0, 
we have that ^p{Xe,^^) > ^p(y,,^^) = which implies {^p(A,,^) > X, > 

X* + ea} 2 {A* + ea<Xe< ye}- This establishes (\M\i . 



Lemma 96 Let = s — jx. Then, under the constraint that limits are taken with |^ fixed, 

4-1 s 
lim V Pr{i:>^ = 1} = 0, lim V nePT{De = 0} = (172) 

£a — ^0 £a — ^0 

for X G (0,oo). Moreover, lime^^o?^4 Pr{£>4 = 0} = if Cj^ > r(A). 



Proof. Throughout the proof of the lemma, we restrict Ea to be small enough such that < 
'/"^'•'^^ N . For simplicity of notations, let ae = lim^ _j.o ye and be = lim^ _5.o ze. The proof consists 
of three main steps as follows. 

First, we shall show that (|172p holds for A G (0, A*]. By the definition of i^, we have > 
Cs-e^+i. Making use of the first four statements of Lemma [95l we have that zg < ^'^''^^-^ < A for 
all £ < ^£ — 1 and ys-i > ^ ^2"^^ ^ ^* sufficiently small. By the last statement of Lemma 

[95] and using Lemma \3T\ we have 

Fr{De = 1} = Pr{A^ < ze} + Pi{Xe > ye} < Pr{A^ < ze} + Pr {a^ > 

< pra<A±tiUprlA,>^::±^~ 



< exp ( ne.^p [ , ^ I I + exp I ne^p I , A 



for all £ < £e — 1 if Ed > is small enough. Noting that be^^i = X*Cj^+i, a^-i = 
X + be,-i _ A + A*C,,+i A* + a,-i _ A* + ^ ^ ^ 
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which are constants independent of £a > 0. Therefore, both ^p( ^^^^^ \ \) and ^p{ ^ '^^^ ^ , A) are 
negative constants independent of £a > 0. It fohows from Lemma [94l that lini£^_yo X^^Li^ "f^e Pr{-D^ = 
1} = 0. 



Similarly, it can be seen from the definition of is that A < Cg-e^-i- Making use of the first 



four statements of Lemma [95l we have that zi > — |^ > A for + 1 < £ < s if is sufficiently 
small. By the last statement of Lemma ESI and using Lemma [511 we have 

Pr{r», = 0} = Pr{z, < A, < y,} < Pr{A, > < Pr (a, > A±^£i±i\ < exp (nf.^p f ^±^£i±i, A 



for + 1 < ^ < s if > is small enough. By virtue of the definition of i^, we have that ft^^+i 
is greater than A and is independent of > 0. In view of this and the fact that Pr{Ds = 0} = 0, 
we can use Lemma [Ml to arrive at lim£^_j.o J2e=e^+i "-fPrlD^ = 0} = 0. 

Second, we shall show that (jl72p holds for A G (A*,oo). As a direct consequence of the 
definition of is-, we have ^ > Cg-e^+i- Making use of the first four statements of Lemma [95l we 
have that y£ > ^^"^'^^ > ^ ^or all £ < — 1 and Zg-i < ^ ^2°'^ if £a is sufficiently small. By the 
last statement of Lemma [95] and using Lemma [311 we have 

Fr{De = 1} = Pr{A^ > ye} + Pr{A^ < ze} < Pr{A^ > y^} + Pr{A^ < 
< Pr <^ > — } +P'ci\i< 



< exp ( n^^p ( ^ , A I I + exp ( n^^p ( ^ , All 

for all £ < ^£ — 1 if > is small enough. By virtue of the definition of is, we have that 
04 _i is greater than A and is independent of £a > 0. Hence, it follows from Lemma [Ml that 

lime^^o Yfi=i Pr{£>, = 1} = 0. 

In a similar manner, by the definition of is-, we have 4^ < C^^i. Making use of the first four 
statements of Lemma l95l we have that yi < ^±^£+1 < A for + 1 < £ < s if Ea is sufficiently 
small. By the last statement of Lemma l95l and using Lemma ISTl we have 

Pr{r>, = 0} = Pr{z, < A, < ye} < Pr{A, < y^} < Pr (a^ < ^±^h+l\ < exp ( n^^p (^l±^h+l^x 



for is + l<i<sii£>0 is small enough. As a result of the definition of is, we have that a^^+i 
is smaller than A and is independent of > 0. In view of this and the fact that Pr{i?s = 0} = 0, 
we can use Lemma [Ml to conclude that lim£^_j.o X]f=4+i "-^ Pr{-Df = 0} = 0. This proves that 
([T72]l holds for A G (A^oo). 

Third, we shall show that liuis^oni^ PrjD^^^ = 0} = if Cj^ > r(A). 

For A E (0, A*) such that Cj^ > r(A), we have ^ < C^-^^ because of the definition of is- 
Making use of the first four statements of Lemma [MJ we have that Zi^ > ^-y^ > A if > is 
small enough. By the last statement of Lemma [95l and using Lemma 1311 we have 
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Pt{D,^ = 0} = Pr{z,, < A,^ <yiA< Pr{A,^ > zej < Pr {a^^ > ^} < exp (n,,^p a)) . 

Since b^^ is greater than A and is independent of > due to the definition of i^, it follows that 
lim,,^on4Pr{£>4 = 0} = 0. 

For A G (A*,oo) such that Cj^ > r(A), we have ^ < C^-^ as a result of the definition of l^- 
Making use of the first four statements of Lemma [951 ^6 have that yi^ < ^i^££ < A if > is 
small enough. By the last statement of Lemma [951 and using Lemma [311 we have 

Pr{D,^ = 0} = Pr{z,, < A,^ <yij< Pr{A,^ < y^l < Pr {^i. < ^} < exp (n^-^^p a)) . 

Since ae^ is smaller than A and is independent of > as a consequence of the definition of 
it follows that lime^_>.o Pr{Z)^^ = 0} = 0. This concludes the proof of the lemma. 

□ 



Finally, we would like to note that the proof of Theorem [51] can be completed by employing 
Lemma [M] and a similar argument as that of Theorem [TBI 



3.7 Proof of Theorem 



As a result of the definitions of k\ and ^(A), we have that > 1 if and only if Cj^ > r(A). To 
prove Theorem I52[ we need some preliminary results. 



Lemma 97 lim 



Proof. First, we shall consider A G (0, A*). Note that 



^p(z, z + e) = —£ + z\ii yl -\ — j = —e + z 



z 2z^ 



By the definition of sample sizes, we have 



lim 



Cs^eHCS) 



ea->0 n^^p(A* + Ea, A*) 



for 1 < ^ < s. It follows that 



(173) 



lim 



.^p(A,A + £a) ^ Cs-i, ln(C5) 



lim 



Cs_4^p(A, A + Ea) 



lim 



X' 



-"-0 -^ + o{el) 



and 



lim Sax 1^ 

Eu^O V A 



>o A^p(A^ + ea,A*) 



, 1 lii(C<5) /A* 

lim EaWT :2 ~~ ^ V X 



179 



We shall next consider A G (A*, oo). Note that 

— z ln(l + e) = £2 [1 — e + o(e)] — 



\ ! + £/ 1 + e 
By (jl73p . we have 



e - y + o(£2 



and 



er™0 7Vm(A, ea,er) 



£r^o ln(C(5) ^p(A* + ea,A 



£r->0 .^p(A* + Ea, A*) £r^0 _^_^o(|£2) 



lim \/ \ni 

er-!>0 



- lim. / AC.-4ln(C^) 



AC,_4ln(C.5) AX~ 

2A* '^l^a/ 



= lim Er 4 ; o - , , , 

er^O W _lL V A 



□ 



Lemma 98 Let U and V be independent Gaussian random, variables with zero means and unit 
variances. Then, for A G (0, oo) such that Cj^ = r(A) and jx > 1, 

lim Pr{Z = 4} = 1 - lim Pr{Z = 4 + 1} = 1 - $ (z^d) , 

£->0 £->0 



lim 



Pr{|A4 -X\>ex,l = 4} + PrjlA^+i - A| > ea, Z = 4 + 1} 

= Fr {U > d} + Ft ^\U + >dy/l + px, U <i^d^, 

where ex = max{ea,SrX}- 



Proof. We shall first consider A G (A*,oo) such that Cj^ = r(A). Since kx = 1, by Statement 
(V) of Lemma [95l we have 



lim 



lim £r J \ni lim = d lim — — — ^ = 0. 



By a similar argument as in the proof of Lemma [20l we can show that 

lim Pr{Z = 4} = 1 - lim Pr{Z = 4 + 1} = lim Pr{A<;, > z^} 

e^O e-!>0 



Pr{|A4 - A| > eA, / = 4} + Pr{|A4+i - A| > ^a, / = 4 + 1} 



lim 

£^0 L 

lini Pr{|A4 - A| > e^A, A^ > z^} + Pr{|A4+i - A| > e,.A, A^ < z^}. 

£— >-0 
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Note that 

Pr{|A4 - A| > SrX, > Zi^} = Pr 
Therefore, 



/Y7 - ^rV^^ie^ /Y7 ^ /Y7 



Pr{|A4 - A| > SA, i = 4} + Pr{|A,^+i - A| > e^, / = 4 + 1} 
^ Pr{|[/| > tf, [/ > 0} + Pr ||C/ + VpaV^I > dy^l + px, C/ < o} = Pr{|C/| > 4 

for A G (A*, 00) such that Cj^ = r(A). 

Next, we shall now consider A G (0, A*) such that Cj^ = r(A). Since kx = 1, by Statement (V) 
of Lemma [95} we have 



lim 



Z£, - A 



lim e, 



'^4 ,■ ^4 - J ,- ^4 - 
lim — = a lim 



-vd. 



Clearly, 

Pr{|A4 -A| >ea, A4 < 24} = Pr 
Therefore, 



A| /n^ A^, - A 24 - A 



Pr{|A4 - A| > SA, / = 4} + Pr{|A4+i - A| > ea, i = 4 + 1} 
Pr{|;7| >d,U< -ud] + Pr + > d^Jl + px, U > -vd^ 

Pr{C/> 4 + Pr||f/ + ^y| > d^l + px, U <vdY 



□ 



Finally, we would like to note that the proof of Theorem [25] can be completed by employing 
Lemma [97] and similar arguments as that of Theorem [16] Specially, we need to restrict £a to be 
small enough such that ^ < For the purpose of proving Statement (III), we need 

to make use of the following observation: 

fprllA- A| > e4 forAG(0,A*], 
Pr{|A-A| |A-A| >e.A}= <^ 11^ I - a; v, J, 

Pr{|A-A| >e^A} for AG (A*, 00) 



Pr{|A, - A| > Ea] = Pr > ^a^^^ , Pr{|A, - A| > e,A} = Pr {|C/,| > e.v^} 



where, according to the central limit theorem, U£ = ^^L_3. converges in distribution to a Gaussian 
random variable U of zero mean and unit variance as — )• 0. 
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K Proofs of Theorems for Estimation of Normal Mean 
K.l Proof of Theorem 



First, we shall show statement (I) which asserts that Pr{|/i — ^| < e} > 1 — 2s(^6. Define 
m = max{ns, [(cts i„^_i^^5)^/e^] }. Then, {^/rn > (ct^ tn^-i,(5)/e} is a sure event and by the 
definition of the sampling scheme, 

Pr{|Xn-/i| >e, n>nj = Pr{|Xm - /i| > e, n > n J < Pr{|Z^ - /i| > e} 

= Pr{|Xm - f^\>£, \/m > {as t„,,_i,f5)/e} 
< Pt I V^\X^ - fi\ > e X ^^^^^^^ 



Pr^ ^'^'""^' >t„.„,c4. (174) 



Note that ^/In{Xm— fi)/a is a standard Gaussian variable and that ^/m{Xm— fj,)/a is independent 
of as because 



Pr { < It > = > Pr < < u, m = m 



oo 
tn—ris 






I 




oo 
m—ns 


f y/m{X„i 


- m) 


I 



N OO 

< u> Pr{m = m} = $(7i) Pr{m = m} = $(i 



and 



Pr < < cr^ < « > = y Pr 1^ u, m = m, < f > 

^ m=ns I J 

y Pr < < u > Prjm = m, ct^ < v\ 



m=ns 
oo 



= ^ <I>(u) Pr{m = m, as <v} = $(n) Pr{CTs < v} 

m=ns 

= Pr{v^(Xm - < u} Vl{^s < v} 

for any u and v. Therefore, y/m.{X^ — ^j)/as has a Student t-distribution of ns — 1 degrees of 
freedom. It follows from (jl74p that 

Pr{|Xn - /i| > e, n > < 2C5. (175) 

By the definition of the sampling scheme, we have {n = n^} C |e > ^'^J-}'^^ | and thus 

Pr{|X„ - Ml > e, n ^ na < Pr (|X„, - /.| > e > ^1^!^ j < Pr | ^'^"^ ~ ^' > = 2C<5 

(176) 
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for £ = 1, • • • ,s — 1. Combining (jl75p and (|176p yields 



s-l 



Pr{|^ - /i| > e} = Pr{|Xn - ^| > e, n > n J + ^ Pr{|Xn - /i| > e, n = < 2sCS, (177) 



^=1 



which imphes that Pr{|A^ — /i| < e} > 1 — 2sC6 for any and a. This proves statement (I). 

Second, we shah show statement (II) which asserts that hm£_>.o Pr{ | /i — /i| < e} = 1 — 2C(5. 
Obviously, linie^o Pr{n < n^} = 0. Hence, lim£_!.o X^|=i Pr{|Xn — //| > e, n = n^} = and 



s-l 



Pr{|/i-//| >e} = Pr{|Xn-/u| >e, n>nJ + ^Pr{|Xn-^| >e, n = n£} 

£=1 

^ Pr{|Xn-;u| >e, n>nj (178) 

as e — )■ 0. By virtue of (|175|) and (|178p . we have limsup£_5.o — /^| < e} < 2^5, which implies 

that 

liminf Pr{|/i - ^1 > e} > 1 - 2C5. (179) 

On the other hand, by (|178p and the fact that lime_^o Prjn ^ ^s} = Ij we have 

Piilfl - ^l\ < e} Pr{|X„-/i| <e, n>n,J = Pr{|Xm-/i| <£, n>nj 

Pi-{\X^- ^i\<e} 

< Pr ( - Ml < < + V)^stn.-i,cs \ + P J (1 + < , 



< Pr ^ ^l^- - ^' < (1 + v)tn.^,J + Pr ( + < e 



as £ —7- 0, where 77 is a positive number. Noting that 



)V£'+^s j I ' 77(2 + 77)(tn.-l,C5)2 

which tends to as e —t- 0, we have 

limsupPr{|Ai-/x| < e} < Pr | ^'^"^ ~ < (1 + r])tn^_^^^s] ■ 
Since the above argument holds for arbitrarily small r/ > 0, we have 

limsupPr{|/i - ;u| < e} < Pr | ^1^*" ~ ^1 < A = 1 _ 2(6. (180) 

e->0 I CTs 'J 

Combing ()179p and p80p yields lim^^o Pi'{|a^ ~ A*| < ^} = 1 ~ 2^5. This proves statement (II). 

Finally, statements (III) and (IV) can be shown by making use of the observation that n < 
(cts ins-i.c^)^/^^ + 'T-s- This completes the proof of Theorem! 
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K.2 Proof of Theorem 1551 



K.2.1 Proof of Statement (I) 

Define Helmert transform 

u,^^, + + 7 + (181) 

a- + 1) y/i 

for i = 1,2,- •• , oo. Clearly, the Ui are independent Gaussian variables with zero mean and 
variance unity. Since the transformation from (Ui, • • • , Ui) to (Vi, • • • , Vi-i, Wi) is orthogonal for 
any i > 2, the Vi are independent Gaussian variables with zero mean and variance unity. It is 
easily seen that ^/n(Xn - /u)/(t = W„ and Sn = (T^(Er=i - W^) = a^{V^ + ■ ■ ■ + V^_^) for 
n = 2, 3, • • • , oo. Hence, by the definition of the sampling scheme, we have that {\Xn — /i| > e} 
is independent of {n = n} for any n £ It follows from such independency and the definition 
of the sampling scheme that 



= ^Pr{|X„-Ai|>£}Pr{n = n} = 2^ 



1 - $ 



Pr{n = n}. 



This proves statement (I). 



K.2. 2 Proof of Statement (II) 

Define Zj = ^^j-^+^^j for j = 1, 2, • • • , oo, where Vi are defined in p81|) . It is easy to see that Zj 
are identical and independent exponential random variables with density e~^. By the definition 



of cr^, we have = J -^^^ = cry — ' , £ — 1, - ■ ■ ,s and thus 



(ct s tn^-lYs)^ 
^ — > ?1 



^Zj>c^, n>ns. (183) 



It follows from (jl82p and the definition of the stopping rule that 

{n > n^} = I ^ Zj > bi for 1 < i < ^1 (184) 

for i = 1, • • • ,s. Making use of (jl84p and Theorem 1541 we have 

Pr{n > ne} = He{a) (185) 
for i = 1, • • • ,s. Similarly, it follows from (I183|) and the definition of the stopping rule that 

Zj > c, ^Zj > bi ior 1 < i < s\ (186) 
j=i i=i I 
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for n>ns. Making use of (jl86|) and Theorem 1541 we have 



Pr{n > n} = i?*(cj,n) (187) 

for n > Ug. By virtue of (jl85|) . we have Pr{n = rii} = 1 — Pr{n > 77,1} = Ho{a) — Hi{a) and 
Pr{n = Ui} = Pr{n > — Pr{n > ni} = i?£_i(cr) — Hi{a) for \ < i < s. In a similar manner, 

using (|187p . we have Pr{n = n} = Pr{n > n — 1} — Pr{n > n} = H*{a,n — 1) — H*(a,n) for 
n > Ug. This completes the proof of statement (II). 



K.2.3 Proof of Statement (III) 

By the established statement (I), we have 



Pr{|/l-/i| >e} = 2 ^ 



71 <m 



1 - $ 



a 



Pr{n = n} + 2 ^ 



n>m 



1 - ^ 



Note that 



E 



1 - $ 



Pr{n = n} < ^ 



1 - $ 



e\frn 



Pr{n = n} = 



cr 



1 - $ 



Pr{n = 77}. (188) 



e^fm 



Pr{n > 77i} 



< 



< 



1 - $ 
1 - $ 
1 - $ 



cr 
6 



2 ^ m(7t^ - 1)£^ 

m{ns — l)e^ 
(a in,-i,C5)^ 



( ks - 1, 



(a iri,-l,C5)' 



(189) 



for any a € [o, where Xn^-i represents a chi-square random variable of — 1 degrees of 
freedom. Observing that Hi{(t) is monotonically increasing with respect to cr G [a, h] for Q < I < s 
and that H*{a,n) is monotonically increasing with respect to cr G [a,b] for ?7 > n^, we have 
P„ < Pr{n = n} < P„ for o" G [a, 6]. Therefore, 



1 _ $ 



n<.m n<.m 



Pn (190) 



for o- E [a, 6]. So, statement (III) follows from (fTHHIl . (fT89D and (fT90D . 

K.2.4 Proof of Statement (IV) 

Applying ([185]) and (fTHTl) . we have 



E[nl 



s-l 

m + y^(n£+i 



n^) Pr{n > n^} + Pr{n > n} 

n=ns 

00 

ni + E("^+i - rii)Hi{a) + ^ if*(cj, 



s-l 



,n 



(191) 



185 



and 

s—l m oo 

E[n] =ni+^(n^+i-n^)i?f(CT)+ ^ i7*(cj,n)+ ^ Pr{n > n}. (192) 

t=\ n=ns n=m+l 

Note that 



Pr{n > 7^} < Pr > ^^J^f^^} = {xl-i > (^s - l)n7} < 



j^^e-(n7-i) 



for n > m, where the last inequahty can be deduced from Chernoff bounds. Therefore, 



^ Pr{n > n} < — ^ 5(^7) 7, 

n=m+l ^ n=m+l 

where we have introduced function g{x) = (xe~^)" for simphcity of notations. Note that g{x) is 
monotonicahy decreasing with respect to x greater than 1 because g'{x) = vg{x) — l) < for 
X > 1. Making use of the assumption that 717 > rwy > 1 and the monotone decreasing property 
of g{x), we have 

2^ g(n7) 7 < / 9{x)dx = / —i-dX, 

where 



< inf e^'^Efe-'^^] = e"™"^ frnv-fey ^ ^-rav^^ y 
h>0 \ V J 

with K representing a Poisson random variable with mean mvy. It follows that 

n=m+l 

Using inequality v\ < \/27rv v'"e~"~^'^^ , we have 



y Pr{n> n} < ±V2^.-e-'+i^ (HTLY ^-(^.-2). ^ I ^^^y,-im,-i).+^ < ^(m7eri_ 

z — / \ V / 7 V w 7vf e™''''' 

n=m+l ' ' 'V 

(193) 

So, the proof of statement (IV) can be completed by combining ()19ip . ()192p and ()193p . 

K.3 Proof of Theorem [56] 

By (fT75]l and (fTTTl) . we have 

Pr{|/2 - ^1 > e} < 2C(5 + ^ Pr{|Xn - /i| > e, n = ng}. (194) 

£=1 



186 



By the definition of the sampling scheme, we have 

s-i s-i r 



e=i 

s-l 

E 



t. 



EPr 



1 - 5p fc, - 1 



Tii ki 



(ct tn^-ixs)^ 



(195) 



and 



s~2 



< Pr{\X,,,-^i\>e} + J2P^{\Xn,^,-fi\>e, > 



s-2 



1 - $ 



1 - $ 



Ml>e} 


+ EPr{l^»,+i 




s-2 

+ 2E 


1 - $ 










s-2 

+ 2E 


1 - $ ^ 



*nf-l,C<5 J 



fcf - 1, 



fcf 



(196) 



Combining ()194p and (|195p yields 

Pr{|Ai-^|>e}<2C5 + E 



s-l 



1 - 5p fc, - 1 



(197) 



where the upper bound in the right side of (jl97p monotonically decreases from s — 1 + 2(^5 to 2^5 
as a increases from to oo. Since < C < |, there exists a unique number a such that 



s-l 

E 



1- Spike -I, 



ne ki 



(o" tns-l,C&f 



(1 - 205 



and that Pr{|/1 — /i| > e} < 5 for o" > o". On the other hand, combining (jl94p and (jl96p yields 

s-2 r 



1 - $ 



/?7.1 



-2E 



1 - $ 



- 1, 



rif kg £^ 



(o- i„^_i,^i-)2 



, (198) 



where the upper bound in the right side of p98p monotonically increases from 2C,5 to s — 1 + 2C,5 
as a increases from to oo. Since < ^ < ^, there exists a unique number g_ such that 



1 _ $ 



s-2 

^=1 



1 - $ 



5p /cf - 1, 



n£ ki 



and that Pr{|/x — /i| > e} < 5 for a < a. This completes the proof of Theorem [561 
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K.4 Proof of Theorem [57] 

By the definition of the stopping rule, we have 



Pr{|/i-/i| > e|^|} < ^Pri l/i^-^l >e|/i|, > 

£=1 

By virtue of identity ([1]), we have 



l + -]o-e 



111 \ £ 



I l + Sgn(M^)£ ^Ai7 V \ 

+ Pr (a^ > ^^W- IM.I > ^ii^ f 1 + -) 

[ l + sgn(/x^)£ ^iil V £/ 

+ Pr (a^ - > ^^^f^V, IM.I > ^^^5^ f 1 + -) ^^ 
\ l-sgn(/x^)£ 7717 V £/ J 

< Pr(|/i.-Ml>^>^i=^-. 

1 + £ ^nj 



- > tnf-i, I = 2Cd£ 

for all£>0. Therefore, Pr {|^ - /i| > e|/i|} < 2 = 2(r + l)C-5. 

The finite stopping property of the sampling scheme can be shown by an argument similar to 
the proof of statement (I) of Theorem [23j 

K.5 Proof of Theorem [581 

By the definition of the stopping rule, we have 

Pr{|/i - ^J.\> niax(£a, £^1^1)} < y^Pr (Im^ - mI > max(£a, £r|M|), max ( £„, fllJ/fil J > hlrzliS^S-A . 
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By virtue of identity ([T]), we have 

Pr (l/i^ - yu| > max(ea, £r\l^\), max ( e^, £!-lifii ] > ^»f-i_C^^ - 1 
= Pr < min f /i, - e., t— ^f^) , max L, ^) > ^-^^ A 

+ Pr(M>maxf/i. + ^a, ^ , nm^ ( ej-fp^] > '-^^^^^e] 

= Pi < fj,^- fi> max £„, — ,1! , , max Sa, , > _ ^ 

+ Pr Ul- max Ea, ,1 . , max Ea, , > o"^ ^ 

\ ^ C-Sf I = ^C'Jf 

for aU i>0. Therefore, Pr {j^i - > max{ea, £r\f^\)} < '^TT=i C^t = + 1)C5- 

The finite stopping property of the samphng scheme can be shown by an argument similar to 
the proof of statement (I) of Theorem [231 



L Proofs of Theorems for Estimation Following Tests 
L.l Proof of Theorem I 



Since is a ULE of 0, by virtue of Lemma [Sj we have that Pr{0 < z | ^} is non-increasing 
with respect to 6 no less than z and that Pr{0 > z | ^} is non-decreasing with respect to 9 no 
greater than z. This implies that Pr{0 < z \ 9} is non-increasing with respect to G © and 
that Pr{0 > z I ^} is non-decreasing with respect to G G. By the definitions of Fg{z,9) and 
Gg{z,9) given in Section [2.51 we have that F^{z,9) is non-increasing with respect to ^ € and 
that Gg{z,9) is non-decreasing with respect to 9 £ @. Recalling the definition of ^{0,n), we 
have that ^(0, n)) < |} is a sure event and consequently 



{0 > ^(0,n)} = > ^ie,n), Fg(0,^(0,n)) < -| C |0 > ^(0,n), Fg(0,0) < -| C |Fg(0,(?) < - 
which implies that 

Pr{0 > ^(?, n)} < Pr |Fg(0, 0) < |1 (199) 



where the last inequality follows from Lemma [2j On the other hand, recalling the definition of 
^{6, n), we have that {Gg(0, ^(0, n)) < f} is a sure event and consequently 



{e < ^{e,n)} = i^e< if(0,n), Gg(0,^(0,n)) <-}c^e< J^{e,n), G^{e,e) <-\q{ G^{e,e) < 

which implies that 

Fv{9 < ^(6, n)} < Pr <j G^{d, ^) < ^ I < ^, (200) 
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where the last inequahty follows from Lemma [2j Finally, by vntue of (jl99p and (j200p . we have 
Pr{^(e, n) <e <'^(e,n)\6}>l- Fv{9 > '^{d, n)} - Fr{e < ^{6, n)} > 1 - - - - = 1 - J. 
This completes the proof of the theorem. 



L.2 Proof of Theorem 

Since p is a ULE of p € Q, by Lemma [3l we can show that that Pr{p < z \ p} is non-increasing 
with respect to p G @ and that Pr{p > z \ p} is non-decreasing with respect to p E Q. Define 
cumulative distribution functions 

{Pt{p < z I p} for p e 8, | Pr{p > z \ p} for p e 9, 

1 forp<0, Gp{z,p)^}o forp<0, 

for p > 1 [ 1 for p > 1 

where z assumes values from the support of p. Then, F^{z,p) is non-increasing with respect 
to p £ @ and that Gp{z,p) is non-decreasing with respect to p £ @. Recalling the definition of 
(p, n), we have that {F^{p, ^ {p, n) + -^) < |} is a sure event and consequently {p > ^{p, n)} = 
{p > ^(p,n) + ^, Fp(p,^(p,n) + i) <l}C{p> ^(p,n) + ^, F^{p,p) < |} C < §}, 

which implies that 

Pr{p> ^(p,n)} <Pr|Fp(p,p) < ^} < ^, (201) 

where the last inequality follows from Lemma [2j On the other hand, recalling the definition of 
^(p, n), we have that {Gp{p, ^(p, n) — ■^) < |} is a sure event and consequently {p < J^{p, n)} = 
{p < if(p,n) - Gg(p,if(p,n) - ^) < f} C {p < J^{p,n) - G^{p,p) < |} C {G^{p,p) < f}, 
which implies that 

Pr{p < ^(p,n)} < Pr |Gp(p,p) < ^} < ^, (202) 

where the last inequality follows from Lemma [2l Finally, by virtue of ()20ip and (j202p . we have 

6 S 

Pr{^(p, n) < p < '^(p, n) I p} > 1 - Pr{p > ^(p, n)} - Pr{p < ^(p, n)} > 1 - - - - = 1 - 5. 
This completes the proof of the theorem. 



L.3 Proof of Theorem | 

Note that 

Pv{^(\i,ne) < A < ^(Xi,ne) \ A} 
> Pr I if (A,, n,) < A < ^(A,, n,), U (Xe, ne,j-]>X*\X 



2s 



Pr <{ L ( A^, n^, <X<U (^e, n^, , U (^e, n^, ^ ) > A* | A 
PrU(A„n4)<A<f/(A„n„A)|A|>l-A 
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for any A G [A*,oo). Therefore, 

Pr{A^ (^(A,,n,), ^(A,,n,)), Z = £|A}<Pr{A^ (^(A,,n,), '^(A,,n,)) I a} < - 
for i = 1, - ■ ■ ,s and any A G [A*, oo). It follows that 

s 

Pr{A^ (if(A,n), -^(A,!!)) |A} = ^Pr{A^ (^(Af,n,), '2r(A^,n^)), Z = £ | a} < 5 
for any A G [A*,oo). The theorem immediately follows. 



M Proof of Theorem [64 

Note that 

Pr{if(A,,n,) < A<^(A,,n,) | A} 
> Pr I if (A,, n,) < A < ^(A,, n,), U (le, n,, > A* | A 

= Prji (^e,ne,-^^ <X<U (^A£,n^,^^ , U (^A^.n^,^^ > A* | A 

= Pr {l (a„ n, A) < a < f/ (a„ n„ ^) I a} > 1 - ^ 

for any A G [A*,oo). The theorem immediately follows. 

N Proofs of Theorems for Multistage Linear Regression 

N.l Proof of Theorem [65] 

By the definition of the stopping rule, 

Pr{|3,-ft| < J^Pri |B,,^-/3i| >ei >t„,_^,(;5, 



oo 



for z = 1, • • • , m. From the classical theory of linear regression, we know that Bi^£—Pi is a Gaussian 
random variable of zero mean, variance cj^ [(XJX^)"^]^^ and that {n£ — m){^)'^ is a chi-square 
variable of — m degrees of freedom. Moreover, Bi^i — j5i is independent of (n^ — m){^)'^ . It 
follows that {Bix — ft) possesses a Student t-distribution of — m degrees 

of freedom. Hence, by (|203p . we have 

oo 

Pr{|3,- >£,} <2j^C5f = 2(t + 1)C<5 (204) 
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for i = 1, - ■ ■ ,m. By the definition of the stopping rule, 



FT{\a-a\>e} < 2^Pi' <\(Ti - a\ > e, . — (Ti-e<ai< — (Ti + e 

e=l I V -^^^e-iri; CSi V ^n^-m, l-C^i , 



oo 

< 

oo 



E l ^ ll£ — III ^ ^ 
Fv <o-i - a < -e, . — o-g - e < a 





— m 






/ T 


— m 




-m, CSi 



E„ I ^ lie — III ^ ^ 
Fr <ai - a > e, J— a-i + e> ae 

\ y ^ne—m, 1— C''^ 



=1 



< 



=1 



^ni—m, (St ) £—1 y ^n£—m, 1—(^S, 



Recalling that (n^ — is a chi-square variable of ni — m degrees of freedom, we have 



p I — —^e<(j\<CSi, ^AJ^^ — ^S-^>ctI<C5^ (206) 
for alU > 0. Combining (1205]) and ([2061) yields 

oo 

Fi{\a -a\>e}<2'^C6e = 2(r + 1)C6. (207) 

£=1 

By virtue of and (pOT]) . we have 

Pr{|o^ — it| < e, |/3j — < ffj for i = 1, • • • , m} 

m 

> l-^Pr{|3,-ft| >ea-Pr{|^-a| >£} 

i=l 

> 1 - 2m(r + 1)C(^ - 2(r + 1)C5 = 1 - 2(m + l)(r + 1)CS. 

The finite stopping property of the sampling scheme can be shown by an argument similar to 
the proof of statement (I) of Theorem [23j This completes the proof of the theorem. 



N.2 Proof of Theorem I 

By the definition of the stopping rule, 



Pr{|3, - > e.m} < ^ Fr - A| > cs. ^[(XjX^)^ < ^^^} 



(208) 
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for i = 1, - ■ ■ ,m. By identity ([T|), we have 



\Bi 



Pr <^ A < 



Bi 



1 + sgn(Si,^)e, 



Pr <^ Bi^e - ft > 



1 - sgn{Bi^i)ei 



e.. 



+ Si 



\Bi,e 
\B 



1 + sgn{Bi^i)ei 



B,^[{x]x,r%<- 



Si 



1 - sgn{Bi^e)ei v l j»« i 



\Bi 



\B, 



< Fr \ \Bi^i - /3i\> 



l + Si 



> t 



ni-m, C<5f 



< Pr 



> t 



aeJ[{x}x,y^] 



rti—m, C,5i 



= 2C5i (209) 

for z = 1, • • • , m, where the last equality (|209|) follows from the fact that {Bi^i—j3i) ^^ [{X\X e)^^] .^ 
possesses a Student t-distribution oin^ — m degrees of freedom. Combining ()208p and ()209p yields 



Pr{|A - ftl > e^lftl} < 2 ^C<5, = 2(r + l)Cc^ 



(210) 



for i = 1, • • • , m. By the definition of the stopping rule, 



Pr{|5^ - cr| > £cr} < ^ Pr i - cr| > ecr, 



< m — m < 



Pr < < (1 - e)cr < a\ 



ni — m 



(l + £)2 - - (l-e)2 

Pr <! CTf > (1 + £)cr > cr 



Hi ~ m 



< 



E 



2(t + l)a 



ne — m 



CT^ < (T ^ + Pr 



I ne — m 



(Ti > a 



(211) 



where (j21ip follows from an argument similar to that of (j206p . Making use of (j210p and (j21ip . 
we have 

Pr{|CT - o-| < ea, |3j - ftl < ^ilftl for i = 1, • • • ,m} 

m 

> 1 - J^Pr{|3, - Al > em} - - ct| > ea} 

i=l 

> 1 - 2m(r + 1)CS - 2(r + 1)C(^ = 1 - 2(m + l)(r + 1)C5. 

The finite stopping property of the sampling scheme can be shown by an argument similar to the 
proof of statement (I) of Theorem [23j This completes the proof of the theorem. 



193 



O Proofs of Theorems for Estimation of Quantile 



O.l Proof of Theorem ET] 

By the definition of tfie stopping rule, 

oo 

M\ip-^p\>e}<J2Pr{\ip.i~^p\>e, X,,:„, -e<|p, <X,,:„,+£}, (212) 
i=i 

where 

< Pr {ip < Ip,i - £ < X^,..n,} + Pr {Cp > Ip,i + e > X,,..n,} 

< Pr {X,,:„, > ^p} + Pr {Xj,.,n, < ip} (213) 

for ah i>0. 

Now, let denote the number of samples among Xi, • • • , which are no greater than ^p. 
Then, {Xi^.,n, > ^p} ^ {Ki < k} and thus Pr > ^p} < Pr{i^£ < k} = EICq {"k)[Px{ip)f[l- 

Fx{i.p)T''-^ . By the definition of ^p, we have Fx{ip) > p. Making use of the fact that XlLo Q^'^i'^- 
Q^n~k jg nionotonically decreasing with respect to G (0, 1), we have that 

Pr > < i^^y^l - Pr^' < CSi, (214) 

where the last inequality follows from the definition of i£. On the other hand, let denote 
the number of samples among Xi,--- ,Xni which are smaller than ^p. Then, {Xj^-m < (,p} ^ 
{K* > je} and thus Pr{X,,.„, < ^p} < Pr{K* > je} = ElLj, (10 " F^iCpT'-", 

where Fxi^p) ~ < Cp}- By the definition of ^p, we have F^{£,p) ^ P- Making use of the fact 

that Ylk=m (fc)^'^(^ ~ 9)^~^ is monotonically increasing with respect to ^ € (0, 1), we have that 

Pr{X,,.„, < U < E /(I -P^"' < C<5^> (215) 

where the last inequality follows from the definition of j^. Combining ()212p . ()213p . ()214p and 
(f2T5]l yields Pr{|^p - ^p| > e} < 2YT^i = 2(t + The finite stopping property of the 

sampling scheme can be shown by an argument similar to the proof of statement (I) of Theorem 



0.2 Proof of Theorem 

By the definition of the stopping rule. 



Pr{|ip - ip\ > £\^p\} < E M\ip,i - ^p\ > £\^p\, [1 - sgn(ip,i)e]Xj,.,n, < < [1 + sgn(ip_^)e]X,,.„ J. 



(216) 
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By identity ([T]), we have 

- < ^^'!^ , ' [1 ~ sgn(ip_^)e]Xj,:„, < ip,^ < [1 + sgn(ip^^)e]X,,.,„, \ 

[ l + sgn(|p_^)£ J 

+ i > ^^j^~r' " sgn(ip^^)e]Xj,:„, < ip_^ < [1 + sgn(ip_^)e]X,,:„, i 

[ 1 - sgn{tj^j,)e J 

- < ^ — ^ ^'^-^ [ + Pr I > - — > I 

[ l + sgn(|p_f)£ J [ l-sgn(|p.^)£ J 

< Pr{X,,:„, > ^p} + Pt{X„.,„, < Cp} (217) 

for all £>0. Combining ^1^, ^15^, ^W^i and ^17^ yields Fr{\ip - Cp| > < 2 E^i C-^^ = 

2(r + 1)C^- The finite stopping property of the sampling scheme can be shown by an argument 
similar to the proof of statement (I) of Theorem [23l 

0.3 Proof of Theorem [69] 

By the definition of the stopping rule and identity ([1]), we have 
Prjl^p-Cpl > max(ea, er|Cp|)| 

oo 

i=i 

Xj^:ni, - max(ea, sgn(^p (;)erXj^.nJ < ?p,^ < Xi^:ni + max(ea, Sgn(^p 



oo ( / 



y]Pr <! > max ( 1^^^ + e^, ^^'1 ^ ) , Xj,:n, - max(ea, sgn(^p^^)e^Xj^:„J < 1^, 



f;Prhp<min(4,-e,, ^^'1 ^ | < X, 

I V l + sgn(^p,,Ky 



oo 



+ y]Pr i > max (^pg + ^^^^ 

£=1 [ \ l-sgn(^p, 



^=1 



where the last inequality follows from (j214p and (j215p . The finite stopping property of the 
sampling scheme can be shown by an argument similar to the proof of statement (I) of Theorem 
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